Compare commits

..

30 Commits

Author SHA1 Message Date
Igor Loskutov
0d4c5c463c feat: standalone frontend uses production build instead of dev server
Override web service in docker-compose.standalone.yml to build from
www/Dockerfile (multi-stage: deps → build → standalone runner) instead
of running pnpm dev with bind-mounted source.
2026-02-12 11:27:45 -05:00
Igor Loskutov
f6a23cfddd fix: standalone GPU service connectivity with host network mode
Server runs with network_mode: host and can't resolve Docker service
names. Publish cpu port as 8100 on host, point server at localhost:8100.
Worker stays on bridge network using cpu:8000. Add dummy
TRANSCRIPT_MODAL_API_KEY since OpenAI SDK requires it even for local
endpoints.
2026-02-11 18:10:20 -05:00
Sergey Mankovsky
b1405af8c7 Remove turbopack 2026-02-11 23:15:32 +01:00
Sergey Mankovsky
71ad8a294f Fix webrtc connection 2026-02-11 23:11:46 +01:00
Sergey Mankovsky
bba272505f Enable server host mode 2026-02-11 23:11:31 +01:00
Igor Loskutov
67aea78243 fix: mock Celery broker in idle transcript validation test
test_validation_idle_transcript_with_recording_allowed called
validate_transcript_for_processing without mocking
task_is_scheduled_or_active, which attempts a real Celery
broker connection (AMQP port 5672). Other tests in the same
file already mock this — apply the same pattern here.
2026-02-11 16:26:24 -05:00
Igor Loskutov
2d81321733 fix: processing page auto-redirect after file upload completes
Three fixes for the processing page not redirecting when status becomes "ended":

- Add useWebSockets to processing page so it receives STATUS events
- Remove OAuth2PasswordBearer from auth_none — broke WebSocket endpoints (500)
- Reconnect stale Redis in ws_manager when Celery worker reuses dead event loop
2026-02-11 15:53:21 -05:00
Igor Loskutov
8c2b720564 fix: improve port conflict detection and ollama model check in standalone setup
- Filter OrbStack/Docker Desktop PIDs from port conflict check (false positives on Mac)
- Check all infra ports (5432, 6379, 3900, 3903) not just app ports
- Fix ollama model detection to match on name column only
- Document OrbStack and cross-project port conflicts in troubleshooting
2026-02-11 14:17:19 -05:00
Sergey Mankovsky
88e945ec00 Add hatchet env vars 2026-02-11 20:02:29 +01:00
Igor Loskutov
f6201dd378 fix: set source_kind to FILE on audio file upload
The upload endpoint left source_kind as the default LIVE even when
a file was uploaded. Now sets it to FILE when the upload completes.
2026-02-11 13:37:55 -05:00
Igor Loskutov
9f62959069 feat: standalone uses self-hosted GPU service for transcription+diarization
Replace in-process pyannote approach with self-hosted gpu/self_hosted/ service.
Same HTTP API as Modal — just TRANSCRIPT_URL/DIARIZATION_URL point to local container.

- Add gpu/self_hosted/Dockerfile.cpu (GPU Dockerfile minus NVIDIA CUDA)
- Add S3 model bundle fallback in diarizer.py when HF_TOKEN not set
- Add gpu service to docker-compose.standalone.yml with compose env overrides
- Fix /browse empty in PUBLIC_MODE (search+list queries filtered out roomless transcripts)
- Remove audio_diarization_pyannote.py, file_diarization_pyannote.py and tests
- Remove pyannote-audio from server local deps
2026-02-11 13:37:55 -05:00
Igor Loskutov
0353c23a94 feat: add local pyannote file diarization processor
Enables file diarization without Modal by using pyannote.audio locally.
Downloads model bundle from S3 on first use, caches locally, patches
config to use local paths. Set DIARIZATION_BACKEND=pyannote to enable.
2026-02-11 13:37:12 -05:00
Sergey Mankovsky
7372f80530 Allow reprocessing idle multitrack transcripts 2026-02-11 19:29:29 +01:00
Sergey Mankovsky
208361c8cc Fix event loop is closed in Celery workers 2026-02-11 19:29:23 +01:00
Sergey Mankovsky
70d17997ef Fix websocket disconnect errors 2026-02-11 19:29:16 +01:00
adc4c20bf4 feat: add local pyannote file diarization processor (#858)
* feat: add local pyannote file diarization processor

Enables file diarization without Modal by using pyannote.audio locally.
Downloads model bundle from S3 on first use, caches locally, patches
config to use local paths. Set DIARIZATION_BACKEND=pyannote to enable.

* fix: standalone setup enables pyannote diarization and public mode

Replace DIARIZATION_ENABLED=false with DIARIZATION_BACKEND=pyannote so
file uploads get speaker diarization out of the box. Add PUBLIC_MODE=true
so unauthenticated users can list/browse transcripts.

* fix: touch env files before first compose_cmd in standalone setup

docker-compose.yml references www/.env.local as env_file, but the
setup script only creates it in step 4. compose_cmd calls in step 3
(Garage) fail on a fresh clone when the file doesn't exist yet.

* feat: standalone uses self-hosted GPU service for transcription+diarization

Replace in-process pyannote approach with self-hosted gpu/self_hosted/ service.
Same HTTP API as Modal — just TRANSCRIPT_URL/DIARIZATION_URL point to local container.

- Add gpu/self_hosted/Dockerfile.cpu (GPU Dockerfile minus NVIDIA CUDA)
- Add S3 model bundle fallback in diarizer.py when HF_TOKEN not set
- Add gpu service to docker-compose.standalone.yml with compose env overrides
- Fix /browse empty in PUBLIC_MODE (search+list queries filtered out roomless transcripts)
- Remove audio_diarization_pyannote.py, file_diarization_pyannote.py and tests
- Remove pyannote-audio from server local deps

* fix: allow unauthenticated GPU requests when no API key configured

OAuth2PasswordBearer with auto_error=True rejects requests without
Authorization header before apikey_auth can check if auth is needed.

* fix: rename standalone gpu service to cpu to match Dockerfile.cpu usage

* docs: add programmatic testing section and fix gpu->cpu naming in setup script/docs

- Add "Testing programmatically" section to standalone docs with curl commands
  for creating transcript, uploading audio, polling status, checking result
- Fix setup-standalone.sh to reference `cpu` service (was still `gpu` after rename)
- Update all docs references from gpu to cpu service naming

---------

Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
2026-02-11 12:41:32 -05:00
Sergey Mankovsky
ec4f356b4c fix: local env setup (#855)
* Ensure rate limit

* Increase nextjs compilation speed

* Fix daily no content handling

* Simplify daily webhook creation

* Fix webhook request validation
2026-02-11 16:59:21 +01:00
Igor Loskutov
39573626e9 fix: invalidate transcript query on STATUS websocket event
Without this, the processing page never redirects after completion
because the redirect logic watches the REST query data, not the
WebSocket status state.

Cherry-picked from feat-dag-progress (faec509a).
2026-02-10 20:27:34 -05:00
Igor Loskutov
d9aa6d6eb0 docs: add troubleshooting section + port conflict check in setup script
Port conflicts from stale next dev / other worktree processes silently
shadow Docker container port mappings, causing env vars to appear ignored.
2026-02-10 19:54:04 -05:00
Igor Loskutov
e1ea914675 docs: update standalone md — symlink handling, garage config template 2026-02-10 19:05:02 -05:00
Igor Loskutov
7200f3c65f fix: standalone setup — garage config, symlink handling, healthcheck
- garage.toml: fix rpc_secret field name (was secret_transmitter),
  move to top-level per Garage v1.1.0 spec, remove unused [s3_web]
- setup-standalone.sh: resolve symlinked .env files before writing,
  always ensure all standalone-critical vars via env_set,
  fix garage key create/info syntax (positional arg, not --name),
  avoid overwriting key secret with "(redacted)" on re-run,
  use compose_cmd in health check
- docker-compose.standalone.yml: fix garage healthcheck (no curl in
  image, use /garage stats instead)
2026-02-10 19:04:42 -05:00
Igor Loskutov
2f669dfd89 feat: add custom S3 endpoint support + Garage standalone storage
Add TRANSCRIPT_STORAGE_AWS_ENDPOINT_URL setting to enable S3-compatible
backends (Garage, MinIO). When set, uses path-style addressing and
routes all requests to the custom endpoint. When unset, AWS behavior
is unchanged.

- AwsStorage: accept aws_endpoint_url, pass to all 6 session.client()
  calls, configure path-style addressing and base_url
- Fix 4 direct AwsStorage constructions in Hatchet workflows to pass
  endpoint_url (would have silently targeted wrong endpoint)
- Standalone: add Garage service to docker-compose.standalone.yml,
  setup script initializes layout/bucket/key and writes credentials
- Fix compose_cmd() bug: Mac path was missing standalone yml
- garage.toml template with runtime secret generation via openssl
2026-02-10 18:40:23 -05:00
Igor Loskutov
d25d77333c chore: rename to setup-standalone, remove redundant setup-local-llm.sh 2026-02-10 17:51:03 -05:00
Igor Loskutov
427254fe33 feat: add unified setup-local-dev.sh for standalone deployment
Single script takes fresh clone to working Reflector: Ollama/LLM setup,
env file generation (server/.env + www/.env.local), docker compose up,
health checks. No Hatchet in standalone — live pipeline is pure Celery.
2026-02-10 17:47:12 -05:00
Igor Loskutov
46750abad9 docs: add TASKS.md for standalone env defaults + setup script work 2026-02-10 17:12:01 -05:00
Igor Loskutov
f36b95b09f docs: resolve standalone storage step — skip S3 for live-only mode 2026-02-10 16:48:18 -05:00
Igor Loskutov
608a3805c5 chore: remove completed PRD, rename setup doc, drop response_format tests
- Remove docs/01_ollama.prd.md (implementation complete)
- Rename local-dev-setup.md -> standalone-local-setup.md
- Remove TestResponseFormat class from test_llm_retry.py
2026-02-10 16:14:33 -05:00
Igor Loskutov
d0af8ffdb7 fix: correct PRD goal (demo/eval, not dev replacement) and processor naming 2026-02-10 16:07:16 -05:00
Igor Loskutov
33a93db802 refactor: move Ollama services to docker-compose.standalone.yml
Ollama profiles (ollama-gpu, ollama-cpu) are only for Linux standalone
deployment. Mac devs never use them. Separate file keeps the main
compose clean and provides a natural home for future standalone services
(MinIO, etc.).

Linux: docker compose -f docker-compose.yml -f docker-compose.standalone.yml --profile ollama-gpu up -d
Mac: docker compose up -d (native Ollama, no standalone file needed)
2026-02-10 16:02:28 -05:00
Igor Loskutov
663345ece6 feat: local LLM via Ollama + structured output response_format
- Add setup script (scripts/setup-local-llm.sh) for one-command Ollama setup
  Mac: native Metal GPU, Linux: containerized via docker-compose profiles
- Add ollama-gpu and ollama-cpu docker-compose profiles for Linux
- Add extra_hosts to server/hatchet-worker-llm for host.docker.internal
- Pass response_format JSON schema in StructuredOutputWorkflow.extract()
  enabling grammar-based constrained decoding on Ollama/llama.cpp/vLLM/OpenAI
- Update .env.example with Ollama as default LLM option
- Add Ollama PRD and local dev setup docs
2026-02-10 15:55:21 -05:00
69 changed files with 1101 additions and 5881 deletions

View File

@@ -34,7 +34,7 @@ jobs:
uv run -m pytest -v tests
docker-amd64:
runs-on: [linux-amd64]
runs-on: linux-amd64
concurrency:
group: docker-amd64-${{ github.ref }}
cancel-in-progress: true
@@ -52,14 +52,12 @@ jobs:
github-token: ${{ secrets.GHA_CACHE_TOKEN }}
docker-arm64:
runs-on: [linux-arm64]
runs-on: linux-arm64
concurrency:
group: docker-arm64-${{ github.ref }}
cancel-in-progress: true
steps:
- uses: actions/checkout@v4
- name: Wait for Docker daemon
run: while ! docker version; do sleep 1; done
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build ARM64

2
.gitignore vendored
View File

@@ -23,5 +23,3 @@ www/.env.production
docs/pnpm-lock.yaml
.secrets
opencode.json
vibedocs/

View File

@@ -6,7 +6,7 @@ repos:
- id: format
name: run format
language: system
entry: bash -c 'if [ -f "$HOME/.nvm/nvm.sh" ]; then source "$HOME/.nvm/nvm.sh"; fi; cd www && pnpm format'
entry: bash -c 'cd www && pnpm format'
pass_filenames: false
files: ^www/

View File

@@ -1,43 +1,5 @@
# Changelog
## [0.35.1](https://github.com/GreyhavenHQ/reflector/compare/v0.35.0...v0.35.1) (2026-02-25)
### Bug Fixes
* enable sentry on frontend ([#876](https://github.com/GreyhavenHQ/reflector/issues/876)) ([bc6bb63](https://github.com/GreyhavenHQ/reflector/commit/bc6bb63c32dc84be5d3b00388618d53f04f64e35))
* switch structured output to tool-call with reflection retry ([#879](https://github.com/GreyhavenHQ/reflector/issues/879)) ([5d54758](https://github.com/GreyhavenHQ/reflector/commit/5d547586ef0f54514d1d65aacca8e57869013a82))
## [0.35.0](https://github.com/Monadical-SAS/reflector/compare/v0.34.0...v0.35.0) (2026-02-23)
### Features
* Add Single User authentication to Selfhosted ([#870](https://github.com/Monadical-SAS/reflector/issues/870)) ([c8db373](https://github.com/Monadical-SAS/reflector/commit/c8db37362b6cfd8f772aee8857de2909f283c029))
## [0.34.0](https://github.com/Monadical-SAS/reflector/compare/v0.33.0...v0.34.0) (2026-02-20)
### Features
* add Caddy reverse proxy with auto HTTPS for LAN access and auto-derive WebSocket URL ([#863](https://github.com/Monadical-SAS/reflector/issues/863)) ([7f2a401](https://github.com/Monadical-SAS/reflector/commit/7f2a4013cbb3d3ee3e76885f28d73331dcaf325c))
* add change_seq to transcripts for ingestion support ([#868](https://github.com/Monadical-SAS/reflector/issues/868)) ([d4cc6be](https://github.com/Monadical-SAS/reflector/commit/d4cc6be1fed56ea7fba06acb8d50c9de43b26b07))
* local llm support + standalone-script doc/draft ([#856](https://github.com/Monadical-SAS/reflector/issues/856)) ([b468427](https://github.com/Monadical-SAS/reflector/commit/b468427f1bb12634f5840990e9d64b2c145d7c1a))
* remove network_mode host for standalone WebRTC ([#864](https://github.com/Monadical-SAS/reflector/issues/864)) ([9dbf155](https://github.com/Monadical-SAS/reflector/commit/9dbf155be4de7c059035a75f90c7bf0845344b74))
* standalone frontend uses production build instead of dev server ([#862](https://github.com/Monadical-SAS/reflector/issues/862)) ([5bca925](https://github.com/Monadical-SAS/reflector/commit/5bca92510a5c33f8baeeaac2c346fb1978366ac8))
### Bug Fixes
* auto-rebuild standalone images and blank Hatchet vars ([3d13e5d](https://github.com/Monadical-SAS/reflector/commit/3d13e5d42fc53ce3c005841265ed1e8735a61518))
* check compose version output, not just exit code ([e57c618](https://github.com/Monadical-SAS/reflector/commit/e57c6186f92d66e4525786e56b018c08cf792d2f))
* check for Docker BuildKit (buildx) before building images ([14a8b58](https://github.com/Monadical-SAS/reflector/commit/14a8b5808e5aed860e55aaed35a0fdf8b2f4afa3))
* check for Docker Compose plugin before running standalone setup ([36a8dae](https://github.com/Monadical-SAS/reflector/commit/36a8daee61c2b7a0937fd0914d51fb4ea8212ae7))
* live flow real-time updates during processing ([#861](https://github.com/Monadical-SAS/reflector/issues/861)) ([972a52d](https://github.com/Monadical-SAS/reflector/commit/972a52d22f989f9e2c6f52362b3f1a4e17773663))
* remove max_tokens cap to support thinking models (Kimi-K2.5) ([#869](https://github.com/Monadical-SAS/reflector/issues/869)) ([527a069](https://github.com/Monadical-SAS/reflector/commit/527a069ba9eff6717ccd4bb1e839674edebffceb))
* standalone on ubuntu ([#865](https://github.com/Monadical-SAS/reflector/issues/865)) ([a8ad237](https://github.com/Monadical-SAS/reflector/commit/a8ad237d8571d5ef5c78fb4427c538592d6a7b43))
* standalone server networking and setup diagnostics ([695f3c4](https://github.com/Monadical-SAS/reflector/commit/695f3c49285254869f6a6cbd5f860d1169fa4daa))
## [0.33.0](https://github.com/Monadical-SAS/reflector/compare/v0.32.2...v0.33.0) (2026-02-05)

View File

@@ -0,0 +1,10 @@
# Standalone Compose: Remaining Production Work
## Server/worker/beat: remove host network mode + bind mounts
Currently `server` uses `network_mode: host` and all three services bind-mount `./server/:/app/`. For full standalone prod:
- Remove `network_mode: host` from server
- Remove bind-mount volumes from server, worker, beat (use built image only)
- Update `compose_cmd` in `setup-standalone.sh` to not rely on host network
- Change `SERVER_API_URL` from `http://host.docker.internal:1250` to `http://server:1250` (server reachable via Docker network once off host mode)

View File

@@ -1,25 +0,0 @@
# Reflector self-hosted production — HTTPS via Caddy reverse proxy
# Copy to Caddyfile: cp Caddyfile.selfhosted.example Caddyfile
# Run: ./scripts/setup-selfhosted.sh --ollama-gpu --garage --caddy
#
# DOMAIN defaults to localhost (self-signed cert).
# Set to your real domain for automatic Let's Encrypt:
# export DOMAIN=reflector.example.com
#
# TLS_MODE defaults to "internal" (self-signed).
# Set to "" for automatic Let's Encrypt (requires real domain + ports 80/443 open):
# export TLS_MODE=""
{$DOMAIN:localhost} {
tls {$TLS_MODE:internal}
handle /v1/* {
reverse_proxy server:1250
}
handle /health {
reverse_proxy server:1250
}
handle {
reverse_proxy web:3000
}
}

View File

@@ -1,42 +0,0 @@
# Reflector standalone — HTTPS via Caddy (droplet / IP access)
# Copy to Caddyfile: cp Caddyfile.standalone.example Caddyfile
# Run: docker compose -f docker-compose.standalone.yml --profile ollama-cpu up -d
#
# :443 = catch-all inside container; Docker maps host port 3043 → container 443
# on_demand = generate self-signed cert for IP/SNI on first request (required for bare IP access)
# Browser will warn. Click Advanced → Proceed.
# Access at https://localhost:3043 (or https://YOUR_IP:3043 on droplet)
# Update www/.env.local with: API_URL=https://YOUR_IP:3043, WEBSOCKET_URL=wss://YOUR_IP:3043, SITE_URL=https://YOUR_IP:3043, NEXTAUTH_URL=https://YOUR_IP:3043
:443 {
tls internal {
on_demand
}
handle /v1/* {
reverse_proxy server:1250
}
handle /health {
reverse_proxy server:1250
}
handle {
reverse_proxy web:3000
}
}
# Option B: localhost (comment Option A, uncomment this)
# app.localhost {
# tls internal
# reverse_proxy web:3000
# }
# api.localhost {
# tls internal
# reverse_proxy server:1250
# }
# Option C: Real domain (uncomment and replace example.com)
# app.example.com {
# reverse_proxy web:3000
# }
# api.example.com {
# reverse_proxy server:1250
# }

208
README.md
View File

@@ -34,8 +34,6 @@ Reflector is an AI-powered audio transcription and meeting analysis platform tha
</tr>
</table>
<p align="center" style="font-size: 1.5em; font-weight: bold;">By <a href="https://greyhaven.co">Greyhaven</a></p>
## What is Reflector?
Reflector is a web application that utilizes local models to process audio content, providing:
@@ -46,100 +44,22 @@ Reflector is a web application that utilizes local models to process audio conte
- **Topic Detection & Summarization**: Extract key topics and generate concise summaries using LLMs
- **Meeting Recording**: Create permanent records of meetings with searchable transcripts
## Architecture
Currently we provide [modal.com](https://modal.com/) gpu template to deploy.
The project consists of three primary components:
## Background
- **Back-End**: Python FastAPI server with async database operations and background processing, found in `server/`.
- **Front-End**: Next.js 14 React application with Chakra UI, located in `www/`.
- **GPU Models**: Specialized ML models for transcription, diarization, translation, and summarization.
The project architecture consists of three primary components:
Currently, Reflector supports two input methods:
- **Screenshare capture**: Real-time audio capture from your browser via WebRTC
- **Audio file upload**: Upload pre-recorded audio files for processing
- **Back-End**: Python server that offers an API and data persistence, found in `server/`.
- **Front-End**: NextJS React project hosted on Vercel, located in `www/`.
- **GPU implementation**: Providing services such as speech-to-text transcription, topic generation, automated summaries, and translations.
## Installation
It also uses authentik for authentication if activated.
For full deployment instructions, see the [Self-Hosted Production Guide](docsv2/selfhosted-production.md) and the [Architecture Reference](docsv2/selfhosted-architecture.md).
## Contribution Guidelines
### Self-Hosted Deployment
The self-hosted setup script configures and launches everything on a single server:
```bash
# GPU with local Ollama LLM, local S3 storage, and Caddy reverse proxy
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --garage --caddy
# With a custom domain (enables Let's Encrypt auto-HTTPS)
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --garage --caddy --domain reflector.example.com
# CPU-only mode (slower, no NVIDIA GPU required)
./scripts/setup-selfhosted.sh --cpu --ollama-cpu --garage --caddy
# With password authentication
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --garage --caddy --password mysecretpass
```
The script is idempotent and safe to re-run. See `./scripts/setup-selfhosted.sh --help` for all options.
### Authentication
Reflector supports three authentication modes:
- **Password authentication (recommended for self-hosted / single-user)**: Use the `--password` flag in the setup script. This creates an `admin@localhost` user with the provided password. Users must log in to create, edit, or delete transcripts.
```bash
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --garage --caddy --password mysecretpass
```
- **Authentik OIDC**: For multi-user or enterprise deployments, Reflector supports [Authentik](https://goauthentik.io/) as an OAuth/OIDC provider. This enables SSO, LDAP/AD integration, and centralized user management. Requires configuring `AUTH_BACKEND=jwt` on the backend and `AUTH_PROVIDER=authentik` on the frontend. See the [Self-Hosted Production Guide](docsv2/selfhosted-production.md) for details.
- **Public mode (default when no auth is configured)**: If neither password nor Authentik is set up, Reflector runs in public mode. In this mode, no login is required — anyone with access to the URL can use the application. Transcripts are created anonymously (not tied to any user account), which means they **cannot be edited or deleted** through the UI or API. Anonymous transcripts are automatically cleaned up after 7 days. This mode is suitable for demos or testing but not recommended for production use.
### Development Setup
```bash
# Backend
cd server
uv sync
docker compose up -d redis
uv run alembic upgrade head
uv run -m reflector.app --reload
# In a separate terminal — start the worker
cd server
uv run celery -A reflector.worker.app worker --loglevel=info
# Frontend
cd www
pnpm install
cp .env_template .env
pnpm dev
```
### Modal.com GPU (Optional)
Reflector also supports deploying specialized models (transcription, diarization) to [Modal.com](https://modal.com/) for serverless GPU processing. This is **not integrated into the self-hosted setup script** and must be configured manually.
See [Modal.com Setup Guide](docs/docs/installation/modal-setup.md) for deployment instructions.
## Audio Processing Commands
### Process a local audio file
```bash
cd server
uv run python -m reflector.tools.process path/to/audio.wav
```
### Reprocess an existing transcription
Re-run the processing pipeline on a previously uploaded transcription by its UUID:
```bash
cd server
uv run -m reflector.tools.process_transcript <transcript-uuid> --sync
```
All new contributions should be made in a separate branch, and goes through a Pull Request.
[Conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) must be used for the PR title and commits.
## Usage
@@ -167,9 +87,96 @@ Note: We currently do not have instructions for Windows users.
- Then goto `System Preferences -> Sound` and choose the devices created from the Output and Input tabs.
- The input from your local microphone, the browser run meeting should be aggregated into one virtual stream to listen to and the output should be fed back to your specified output devices if everything is configured properly.
## Installation
*Note: we're working toward better installation, theses instructions are not accurate for now*
### Frontend
Start with `cd www`.
**Installation**
```bash
pnpm install
cp .env.example .env
```
Then, fill in the environment variables in `.env` as needed. If you are unsure on how to proceed, ask in Zulip.
**Run in development mode**
```bash
pnpm dev
```
Then (after completing server setup and starting it) open [http://localhost:3000](http://localhost:3000) to view it in the browser.
**OpenAPI Code Generation**
To generate the TypeScript files from the openapi.json file, make sure the python server is running, then run:
```bash
pnpm openapi
```
### Backend
Start with `cd server`.
**Run in development mode**
```bash
docker compose up -d redis
# on the first run, or if the schemas changed
uv run alembic upgrade head
# start the worker
uv run celery -A reflector.worker.app worker --loglevel=info
# start the app
uv run -m reflector.app --reload
```
Then fill `.env` with the omitted values (ask in Zulip).
**Crontab (optional)**
For crontab (only healthcheck for now), start the celery beat (you don't need it on your local dev environment):
```bash
uv run celery -A reflector.worker.app beat
```
### GPU models
Currently, reflector heavily use custom local models, deployed on modal. All the micro services are available in server/gpu/
To deploy llm changes to modal, you need:
- a modal account
- set up the required secret in your modal account (REFLECTOR_GPU_APIKEY)
- install the modal cli
- connect your modal cli to your account if not done previously
- `modal run path/to/required/llm`
## Using local files
You can manually process an audio file by calling the process tool:
```bash
uv run python -m reflector.tools.process path/to/audio.wav
```
## Reprocessing any transcription
```bash
uv run -m reflector.tools.process_transcript 81ec38d1-9dd7-43d2-b3f8-51f4d34a07cd --sync
```
## Build-time env variables
Next.js projects are more used to NEXT_PUBLIC_ prefixed buildtime vars. We don't have those for the reason we need to serve a customizable prebuilt docker container.
Next.js projects are more used to NEXT_PUBLIC_ prefixed buildtime vars. We don't have those for the reason we need to serve a ccustomizable prebuild docker container.
Instead, all the variables are runtime. Variables needed to the frontend are served to the frontend app at initial render.
@@ -204,22 +211,3 @@ FEATURE_BROWSE=false
# Enable Zulip integration
FEATURE_SEND_TO_ZULIP=true
```
## Contribution Guidelines
All new contributions should be made in a separate branch, and goes through a Pull Request.
[Conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) must be used for the PR title and commits.
## Future Plans
- **Multi-language support enhancement**: Default language selection per room/user, automatic language detection improvements, multi-language diarization, and RTL language UI support
- **Jitsi integration**: Self-hosted video conferencing rooms with no external API keys, full control over video infrastructure, and enhanced privacy
- **Calendar integration**: Google Calendar and Microsoft Outlook synchronization, automatic meeting room creation, and post-meeting transcript delivery
- **Enhanced analytics**: Meeting insights dashboard, speaker participation metrics, topic trends over time, and team collaboration patterns
- **Advanced AI features**: Real-time sentiment analysis, emotion detection, meeting quality scores, and automated coaching suggestions
- **Integration ecosystem**: Slack/Teams notifications, CRM integration (Salesforce, HubSpot), project management tools (Jira, Asana), and knowledge bases (Notion, Confluence)
- **Performance improvements**: WebAssembly for client-side processing, edge computing support, and network optimization
## Legacy Documentation
The `docs/` folder contains an older Docusaurus-based documentation site. These docs are **no longer actively maintained** and may be outdated. For current installation and deployment instructions, refer to the [`docsv2/`](docsv2/) folder instead.

View File

@@ -1,321 +0,0 @@
# Self-hosted production Docker Compose — single file for everything.
#
# Usage: ./scripts/setup-selfhosted.sh --gpu --ollama-gpu --garage --caddy
# or: docker compose -f docker-compose.selfhosted.yml --profile gpu [--profile ollama-gpu] [--profile garage] [--profile caddy] up -d
#
# Specialized models (pick ONE — required):
# --profile gpu NVIDIA GPU for transcription/diarization/translation
# --profile cpu CPU-only for transcription/diarization/translation
#
# Local LLM (optional — for summarization/topics):
# --profile ollama-gpu Local Ollama with NVIDIA GPU
# --profile ollama-cpu Local Ollama on CPU only
#
# Other optional services:
# --profile garage Local S3-compatible storage (Garage)
# --profile caddy Reverse proxy with auto-SSL
#
# Prerequisites:
# 1. Run ./scripts/setup-selfhosted.sh to generate env files and secrets
# 2. Or manually create server/.env and www/.env from the .selfhosted.example templates
services:
# ===========================================================
# Always-on core services (no profile required)
# ===========================================================
server:
build:
context: ./server
dockerfile: Dockerfile
image: monadicalsas/reflector-backend:latest
restart: unless-stopped
ports:
- "127.0.0.1:1250:1250"
- "50000-50100:50000-50100/udp"
env_file:
- ./server/.env
environment:
ENTRYPOINT: server
# Docker-internal overrides (always correct inside compose network)
DATABASE_URL: postgresql+asyncpg://reflector:reflector@postgres:5432/reflector
REDIS_HOST: redis
CELERY_BROKER_URL: redis://redis:6379/1
CELERY_RESULT_BACKEND: redis://redis:6379/1
HATCHET_CLIENT_SERVER_URL: ""
HATCHET_CLIENT_HOST_PORT: ""
# Specialized models via gpu/cpu container (aliased as "transcription")
TRANSCRIPT_BACKEND: modal
TRANSCRIPT_URL: http://transcription:8000
TRANSCRIPT_MODAL_API_KEY: selfhosted
DIARIZATION_BACKEND: modal
DIARIZATION_URL: http://transcription:8000
TRANSLATION_BACKEND: modal
TRANSLATE_URL: http://transcription:8000
# WebRTC: fixed UDP port range for ICE candidates (mapped above)
WEBRTC_PORT_RANGE: "50000-50100"
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_started
volumes:
- server_data:/app/data
worker:
build:
context: ./server
dockerfile: Dockerfile
image: monadicalsas/reflector-backend:latest
restart: unless-stopped
env_file:
- ./server/.env
environment:
ENTRYPOINT: worker
DATABASE_URL: postgresql+asyncpg://reflector:reflector@postgres:5432/reflector
REDIS_HOST: redis
CELERY_BROKER_URL: redis://redis:6379/1
CELERY_RESULT_BACKEND: redis://redis:6379/1
HATCHET_CLIENT_SERVER_URL: ""
HATCHET_CLIENT_HOST_PORT: ""
TRANSCRIPT_BACKEND: modal
TRANSCRIPT_URL: http://transcription:8000
TRANSCRIPT_MODAL_API_KEY: selfhosted
DIARIZATION_BACKEND: modal
DIARIZATION_URL: http://transcription:8000
TRANSLATION_BACKEND: modal
TRANSLATE_URL: http://transcription:8000
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_started
volumes:
- server_data:/app/data
beat:
build:
context: ./server
dockerfile: Dockerfile
image: monadicalsas/reflector-backend:latest
restart: unless-stopped
env_file:
- ./server/.env
environment:
ENTRYPOINT: beat
DATABASE_URL: postgresql+asyncpg://reflector:reflector@postgres:5432/reflector
REDIS_HOST: redis
CELERY_BROKER_URL: redis://redis:6379/1
CELERY_RESULT_BACKEND: redis://redis:6379/1
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_started
web:
build:
context: ./www
dockerfile: Dockerfile
image: monadicalsas/reflector-frontend:latest
restart: unless-stopped
ports:
- "127.0.0.1:3000:3000"
env_file:
- ./www/.env
environment:
NODE_ENV: production
NODE_TLS_REJECT_UNAUTHORIZED: "0"
SERVER_API_URL: http://server:1250
KV_URL: redis://redis:6379
KV_USE_TLS: "false"
NEXTAUTH_URL_INTERNAL: http://localhost:3000
depends_on:
- redis
redis:
image: redis:7.2-alpine
restart: unless-stopped
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 30s
timeout: 3s
retries: 3
volumes:
- redis_data:/data
postgres:
image: postgres:17-alpine
restart: unless-stopped
environment:
POSTGRES_USER: reflector
POSTGRES_PASSWORD: reflector
POSTGRES_DB: reflector
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U reflector"]
interval: 30s
timeout: 3s
retries: 3
# ===========================================================
# Specialized model containers (transcription, diarization, translation)
# Both gpu and cpu get alias "transcription" so server config never changes.
# ===========================================================
gpu:
build:
context: ./gpu/self_hosted
dockerfile: Dockerfile
profiles: [gpu]
restart: unless-stopped
ports:
- "127.0.0.1:8000:8000"
environment:
HF_TOKEN: ${HF_TOKEN:-}
volumes:
- gpu_cache:/root/.cache
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8000/docs"]
interval: 15s
timeout: 5s
retries: 10
start_period: 120s
networks:
default:
aliases:
- transcription
cpu:
build:
context: ./gpu/self_hosted
dockerfile: Dockerfile.cpu
profiles: [cpu]
restart: unless-stopped
ports:
- "127.0.0.1:8000:8000"
environment:
HF_TOKEN: ${HF_TOKEN:-}
volumes:
- gpu_cache:/root/.cache
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8000/docs"]
interval: 15s
timeout: 5s
retries: 10
start_period: 120s
networks:
default:
aliases:
- transcription
# ===========================================================
# Ollama — local LLM for summarization & topic detection
# Only started with --ollama-gpu or --ollama-cpu modes.
# ===========================================================
ollama:
image: ollama/ollama:latest
profiles: [ollama-gpu]
restart: unless-stopped
ports:
- "127.0.0.1:11435:11435"
volumes:
- ollama_data:/root/.ollama
environment:
OLLAMA_HOST: "0.0.0.0:11435"
OLLAMA_KEEP_ALIVE: "24h"
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:11435/api/tags"]
interval: 10s
timeout: 5s
retries: 5
ollama-cpu:
image: ollama/ollama:latest
profiles: [ollama-cpu]
restart: unless-stopped
ports:
- "127.0.0.1:11435:11435"
volumes:
- ollama_data:/root/.ollama
environment:
OLLAMA_HOST: "0.0.0.0:11435"
OLLAMA_KEEP_ALIVE: "24h" # keep model loaded to avoid reload delays
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:11435/api/tags"]
interval: 10s
timeout: 5s
retries: 5
# ===========================================================
# Garage — local S3-compatible object storage (optional)
# ===========================================================
garage:
image: dxflrs/garage:v1.1.0
profiles: [garage]
restart: unless-stopped
ports:
- "3900:3900" # S3 API
- "3903:3903" # Admin API
volumes:
- garage_data:/var/lib/garage/data
- garage_meta:/var/lib/garage/meta
- ./data/garage.toml:/etc/garage.toml:ro
healthcheck:
test: ["CMD", "/garage", "stats"]
interval: 10s
timeout: 5s
retries: 5
start_period: 5s
# ===========================================================
# Caddy — reverse proxy with automatic SSL (optional)
# Maps 80:80 and 443:443 — only exposed ports in the stack.
# ===========================================================
caddy:
image: caddy:2-alpine
profiles: [caddy]
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy_data:/data
- caddy_config:/config
depends_on:
- web
- server
volumes:
postgres_data:
redis_data:
server_data:
gpu_cache:
garage_data:
garage_meta:
ollama_data:
caddy_data:
caddy_config:
networks:
default:
attachable: true

View File

@@ -1,148 +1,11 @@
# Self-contained standalone compose for fully local deployment (no external dependencies).
# Usage: docker compose -f docker-compose.standalone.yml up -d
# Standalone services for fully local deployment (no external dependencies).
# Usage: docker compose -f docker-compose.yml -f docker-compose.standalone.yml up -d
#
# On Linux with NVIDIA GPU, also pass: --profile ollama-gpu
# On Linux without GPU: --profile ollama-cpu
# On Mac: Ollama runs natively (Metal GPU) — no profile needed, services here unused.
services:
caddy:
image: caddy:2-alpine
restart: unless-stopped
ports:
- "3043:443"
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy_data:/data
- caddy_config:/config
depends_on:
- web
- server
server:
build:
context: server
ports:
- "1250:1250"
- "50000-50100:50000-50100/udp"
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- ./server/:/app/
- /app/.venv
env_file:
- ./server/.env
environment:
ENTRYPOINT: server
# Docker DNS names instead of localhost
DATABASE_URL: postgresql+asyncpg://reflector:reflector@postgres:5432/reflector
REDIS_HOST: redis
CELERY_BROKER_URL: redis://redis:6379/1
CELERY_RESULT_BACKEND: redis://redis:6379/1
# Standalone doesn't run Hatchet
HATCHET_CLIENT_SERVER_URL: ""
HATCHET_CLIENT_HOST_PORT: ""
# Self-hosted transcription/diarization via CPU service
TRANSCRIPT_BACKEND: modal
TRANSCRIPT_URL: http://cpu:8000
TRANSCRIPT_MODAL_API_KEY: local
DIARIZATION_BACKEND: modal
DIARIZATION_URL: http://cpu:8000
# Caddy reverse proxy prefix
ROOT_PATH: /server-api
# WebRTC: fixed UDP port range for ICE candidates (mapped above).
# WEBRTC_HOST is set by setup-standalone.sh in server/.env (LAN IP detection).
WEBRTC_PORT_RANGE: "50000-50100"
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_started
worker:
build:
context: server
volumes:
- ./server/:/app/
- /app/.venv
env_file:
- ./server/.env
environment:
ENTRYPOINT: worker
HATCHET_CLIENT_SERVER_URL: ""
HATCHET_CLIENT_HOST_PORT: ""
TRANSCRIPT_BACKEND: modal
TRANSCRIPT_URL: http://cpu:8000
TRANSCRIPT_MODAL_API_KEY: local
DIARIZATION_BACKEND: modal
DIARIZATION_URL: http://cpu:8000
depends_on:
redis:
condition: service_started
beat:
build:
context: server
volumes:
- ./server/:/app/
- /app/.venv
env_file:
- ./server/.env
environment:
ENTRYPOINT: beat
depends_on:
redis:
condition: service_started
redis:
image: redis:7.2
ports:
- 6379:6379
postgres:
image: postgres:17
command: postgres -c 'max_connections=200'
ports:
- 5432:5432
environment:
POSTGRES_USER: reflector
POSTGRES_PASSWORD: reflector
POSTGRES_DB: reflector
volumes:
- ./data/postgres:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -d reflector -U reflector"]
interval: 5s
timeout: 5s
retries: 10
start_period: 15s
web:
image: reflector-frontend-standalone
build:
context: ./www
ports:
- "3000:3000"
command: ["node", "server.js"]
env_file:
- ./www/.env.local
environment:
NODE_ENV: production
# API_URL, WEBSOCKET_URL, SITE_URL, NEXTAUTH_URL from www/.env.local (allows HTTPS)
# Server-side URLs (docker-network internal)
SERVER_API_URL: http://server:1250
KV_URL: redis://redis:6379
KV_USE_TLS: "false"
# Standalone: no external auth provider
FEATURE_REQUIRE_LOGIN: "false"
FEATURE_ROOMS: "false"
NEXTAUTH_SECRET: standalone-local-secret
# Nullify partial auth vars inherited from base env_file
AUTHENTIK_ISSUER: ""
AUTHENTIK_REFRESH_TOKEN_URL: ""
garage:
image: dxflrs/garage:v1.1.0
ports:
@@ -160,6 +23,68 @@ services:
retries: 5
start_period: 5s
ollama:
image: ollama/ollama:latest
profiles: ["ollama-gpu"]
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:11434/api/tags"]
interval: 10s
timeout: 5s
retries: 5
ollama-cpu:
image: ollama/ollama:latest
profiles: ["ollama-cpu"]
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:11434/api/tags"]
interval: 10s
timeout: 5s
retries: 5
# Override server/worker/beat to use self-hosted GPU service for transcription+diarization.
# compose `environment:` overrides values from `env_file:` — no need to edit server/.env.
server:
environment:
TRANSCRIPT_BACKEND: modal
TRANSCRIPT_URL: http://localhost:8100
TRANSCRIPT_MODAL_API_KEY: local
DIARIZATION_BACKEND: modal
DIARIZATION_URL: http://localhost:8100
worker:
environment:
TRANSCRIPT_BACKEND: modal
TRANSCRIPT_URL: http://cpu:8000
TRANSCRIPT_MODAL_API_KEY: local
DIARIZATION_BACKEND: modal
DIARIZATION_URL: http://cpu:8000
web:
image: reflector-frontend-standalone
build:
context: ./www
command: ["node", "server.js"]
volumes: !reset []
environment:
NODE_ENV: production
cpu:
build:
context: ./gpu/self_hosted
@@ -197,45 +122,8 @@ services:
retries: 10
start_period: 120s
ollama:
image: ollama/ollama:latest
profiles: ["ollama-gpu"]
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:11434/api/tags"]
interval: 10s
timeout: 5s
retries: 5
ollama-cpu:
image: ollama/ollama:latest
profiles: ["ollama-cpu"]
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:11434/api/tags"]
interval: 10s
timeout: 5s
retries: 5
volumes:
garage_data:
garage_meta:
ollama_data:
gpu_cache:
caddy_data:
caddy_config:

View File

@@ -13,32 +13,20 @@ cd reflector
./scripts/setup-standalone.sh
```
On Ubuntu, the setup script installs Docker automatically if missing.
The script is idempotent — safe to re-run at any time. It detects what's already set up and skips completed steps.
## Prerequisites
- Docker with Compose V2 plugin (Docker Desktop, OrbStack, or Docker Engine + compose plugin)
- Docker / OrbStack / Docker Desktop (any)
- Mac (Apple Silicon) or Linux
- 16GB+ RAM (32GB recommended for 14B LLM models)
- **Mac only**: [Ollama](https://ollama.com/download) installed (`brew install ollama`)
### Installing Docker (if not already installed)
**Ubuntu**: The setup script runs `install-docker-ubuntu.sh` automatically when Docker is missing. Or run it manually:
```bash
./scripts/install-docker-ubuntu.sh
```
**Mac**: Install [Docker Desktop](https://www.docker.com/products/docker-desktop/) or [OrbStack](https://orbstack.dev/).
## What the script does
### 1. LLM inference via Ollama
**Mac**: starts Ollama natively (Metal GPU acceleration). Pulls the LLM model. Docker containers reach it via `host.docker.internal:11435`.
**Mac**: starts Ollama natively (Metal GPU acceleration). Pulls the LLM model. Docker containers reach it via `host.docker.internal:11434`.
**Linux**: starts containerized Ollama via `docker-compose.standalone.yml` profile (`ollama-gpu` with NVIDIA, `ollama-cpu` without). Pulls model inside the container.
@@ -48,28 +36,28 @@ Generates `server/.env` and `www/.env.local` with standalone defaults:
**`server/.env`** — key settings:
| Variable | Value | Why |
| --------------------- | -------------------------------------------------- | ----------------------------------- |
| `DATABASE_URL` | `postgresql+asyncpg://...@postgres:5432/reflector` | Docker-internal hostname |
| `REDIS_HOST` | `redis` | Docker-internal hostname |
| `CELERY_BROKER_URL` | `redis://redis:6379/1` | Docker-internal hostname |
| `AUTH_BACKEND` | `none` | No Authentik in standalone |
| `TRANSCRIPT_BACKEND` | `modal` | HTTP API to self-hosted CPU service |
| `TRANSCRIPT_URL` | `http://cpu:8000` | Docker-internal CPU service |
| `DIARIZATION_BACKEND` | `modal` | HTTP API to self-hosted CPU service |
| `DIARIZATION_URL` | `http://cpu:8000` | Docker-internal CPU service |
| `TRANSLATION_BACKEND` | `passthrough` | No Modal |
| `LLM_URL` | `http://host.docker.internal:11435/v1` (Mac) | Ollama endpoint |
| Variable | Value | Why |
|----------|-------|-----|
| `DATABASE_URL` | `postgresql+asyncpg://...@postgres:5432/reflector` | Docker-internal hostname |
| `REDIS_HOST` | `redis` | Docker-internal hostname |
| `CELERY_BROKER_URL` | `redis://redis:6379/1` | Docker-internal hostname |
| `AUTH_BACKEND` | `none` | No Authentik in standalone |
| `TRANSCRIPT_BACKEND` | `modal` | HTTP API to self-hosted CPU service |
| `TRANSCRIPT_URL` | `http://cpu:8000` | Docker-internal CPU service |
| `DIARIZATION_BACKEND` | `modal` | HTTP API to self-hosted CPU service |
| `DIARIZATION_URL` | `http://cpu:8000` | Docker-internal CPU service |
| `TRANSLATION_BACKEND` | `passthrough` | No Modal |
| `LLM_URL` | `http://host.docker.internal:11434/v1` (Mac) | Ollama endpoint |
**`www/.env.local`** — key settings:
| Variable | Value |
| ----------------------- | ------------------------------------------ |
| `API_URL` | `https://localhost:3043` or `https://YOUR_IP:3043` (Linux) |
| `SERVER_API_URL` | `http://server:1250` |
| `WEBSOCKET_URL` | `auto` |
| `FEATURE_REQUIRE_LOGIN` | `false` |
| `NEXTAUTH_SECRET` | `standalone-dev-secret-not-for-production` |
| Variable | Value |
|----------|-------|
| `API_URL` | `http://localhost:1250` |
| `SERVER_API_URL` | `http://server:1250` |
| `WEBSOCKET_URL` | `ws://localhost:1250` |
| `FEATURE_REQUIRE_LOGIN` | `false` |
| `NEXTAUTH_SECRET` | `standalone-dev-secret-not-for-production` |
If env files already exist (including symlinks from worktree setup), the script resolves symlinks and ensures all standalone-critical vars are set. Existing vars not related to standalone are preserved.
@@ -79,14 +67,14 @@ Standalone uses [Garage](https://garagehq.deuxfleurs.fr/) — a lightweight S3-c
**`server/.env`** — storage settings added by the script:
| Variable | Value | Why |
| ------------------------------------------ | -------------------- | ------------------------------------- |
| `TRANSCRIPT_STORAGE_BACKEND` | `aws` | Uses the S3-compatible storage driver |
| `TRANSCRIPT_STORAGE_AWS_ENDPOINT_URL` | `http://garage:3900` | Docker-internal Garage S3 API |
| `TRANSCRIPT_STORAGE_AWS_BUCKET_NAME` | `reflector-media` | Created by the script |
| `TRANSCRIPT_STORAGE_AWS_REGION` | `garage` | Must match Garage config |
| `TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID` | _(auto-generated)_ | Created by `garage key create` |
| `TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY` | _(auto-generated)_ | Created by `garage key create` |
| Variable | Value | Why |
|----------|-------|-----|
| `TRANSCRIPT_STORAGE_BACKEND` | `aws` | Uses the S3-compatible storage driver |
| `TRANSCRIPT_STORAGE_AWS_ENDPOINT_URL` | `http://garage:3900` | Docker-internal Garage S3 API |
| `TRANSCRIPT_STORAGE_AWS_BUCKET_NAME` | `reflector-media` | Created by the script |
| `TRANSCRIPT_STORAGE_AWS_REGION` | `garage` | Must match Garage config |
| `TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID` | *(auto-generated)* | Created by `garage key create` |
| `TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY` | *(auto-generated)* | Created by `garage key create` |
The `TRANSCRIPT_STORAGE_AWS_ENDPOINT_URL` setting enables S3-compatible backends. When set, the storage driver uses path-style addressing and routes all requests to the custom endpoint. When unset (production AWS), behavior is unchanged.
@@ -119,25 +107,23 @@ Run automatically by the `server` container on startup (`runserver.sh` calls `al
### 7. Health check
Verifies:
- CPU service responds (transcription + diarization ready)
- Server responds at `http://localhost:1250/health`
- Frontend serves at `http://localhost:3000` (or via Caddy at `https://localhost:3043`)
- Frontend serves at `http://localhost:3000`
- LLM endpoint reachable from inside containers
## Services
| Service | Port | Purpose |
| ---------- | ---------- | -------------------------------------------------- |
| `caddy` | 3043 | Reverse proxy (HTTPS, self-signed cert) |
| `server` | 1250 | FastAPI backend (runs migrations on start) |
| `web` | 3000 | Next.js frontend |
| `postgres` | 5432 | PostgreSQL database |
| `redis` | 6379 | Cache + Celery broker |
| `garage` | 3900, 3903 | S3-compatible object storage (S3 API + admin API) |
| `cpu` | — | Self-hosted transcription + diarization (CPU-only) |
| `worker` | — | Celery worker (live pipeline post-processing) |
| `beat` | — | Celery beat (scheduled tasks) |
| Service | Port | Purpose |
|---------|------|---------|
| `server` | 1250 | FastAPI backend (runs migrations on start) |
| `web` | 3000 | Next.js frontend |
| `postgres` | 5432 | PostgreSQL database |
| `redis` | 6379 | Cache + Celery broker |
| `garage` | 3900, 3903 | S3-compatible object storage (S3 API + admin API) |
| `cpu` | — | Self-hosted transcription + diarization (CPU-only) |
| `worker` | — | Celery worker (live pipeline post-processing) |
| `beat` | — | Celery beat (scheduled tasks) |
## Testing programmatically
@@ -171,89 +157,8 @@ Expected result: status `ended`, auto-generated `title`, `short_summary`, `long_
CPU-only processing is slow (~15 min for a 3 min audio file). Diarization finishes in ~3 min, transcription takes the rest.
## Enabling HTTPS (droplet via IP)
To serve Reflector over HTTPS on a droplet accessed by IP (self-signed certificate):
1. **Copy the Caddyfile** (no edits needed — `:443` catches all HTTPS inside container, mapped to host port 3043):
```bash
cp Caddyfile.standalone.example Caddyfile
```
2. **Update `www/.env.local`** with HTTPS URLs (port 3043):
```env
API_URL=https://YOUR_IP:3043
WEBSOCKET_URL=wss://YOUR_IP:3043
SITE_URL=https://YOUR_IP:3043
NEXTAUTH_URL=https://YOUR_IP:3043
```
3. **Restart services**:
```bash
docker compose -f docker-compose.standalone.yml --profile ollama-cpu up -d
```
(Use `ollama-gpu` instead of `ollama-cpu` if you have an NVIDIA GPU.)
4. **Access** at `https://YOUR_IP:3043`. The browser will warn about the self-signed cert — click **Advanced** → **Proceed to YOUR_IP (unsafe)**. All traffic (page, API, WebSocket) uses the same origin, so accepting once is enough.
## Troubleshooting
### ERR_SSL_PROTOCOL_ERROR when accessing https://YOUR_IP
You do **not** need a domain — the setup works with an IP address. This error usually means Caddy isn't serving TLS on port 3043. Check in order:
1. **Caddyfile** — must use the `:443` catch-all (container-internal; Docker maps host 3043 → container 443):
```bash
cp Caddyfile.standalone.example Caddyfile
```
2. **Firewall** — allow port 3043 (common on DigitalOcean):
```bash
sudo ufw allow 3043
sudo ufw status
```
3. **Caddy running** — verify and restart:
```bash
docker compose -f docker-compose.standalone.yml ps
docker compose -f docker-compose.standalone.yml logs caddy --tail 20
docker compose -f docker-compose.standalone.yml --profile ollama-cpu up -d
```
4. **Test from the droplet** — if this works, the issue is external (firewall, network):
```bash
curl -vk https://localhost:3043
```
5. **localhost works but external IP fails** — Re-run the setup script; it generates a Caddyfile with your droplet IP explicitly, so Caddy provisions the cert at startup:
```bash
./scripts/setup-standalone.sh
```
Or manually create `Caddyfile` with your IP (replace 138.197.162.116):
```
https://138.197.162.116, localhost {
tls internal
handle /v1/* { reverse_proxy server:1250 }
handle /health { reverse_proxy server:1250 }
handle { reverse_proxy web:3000 }
}
```
Then restart: `docker compose -f docker-compose.standalone.yml --profile ollama-cpu up -d`
6. **Still failing?** Try HTTP (no TLS) — create `Caddyfile`:
```
:80 {
handle /v1/* { reverse_proxy server:1250 }
handle /health { reverse_proxy server:1250 }
handle { reverse_proxy web:3000 }
}
```
Update `www/.env.local`: `API_URL=http://YOUR_IP:3043`, `WEBSOCKET_URL=ws://YOUR_IP:3043`, `SITE_URL=http://YOUR_IP:3043`, `NEXTAUTH_URL=http://YOUR_IP:3043`. Restart, then access `http://YOUR_IP:3043`.
### Docker not ready
If setup fails with "Docker not ready", on Ubuntu run `./scripts/install-docker-ubuntu.sh`. If Docker is installed but you're not root, run `newgrp docker` then run the setup script again.
### Port conflicts (most common issue)
If the frontend or backend behaves unexpectedly (e.g., env vars seem ignored, changes don't take effect), **check for port conflicts first**:
@@ -271,7 +176,6 @@ lsof -ti :3000 | xargs kill
```
Common causes:
- A stale `next dev` or `pnpm dev` process from another terminal/worktree
- Another Docker Compose project (different worktree) with containers on the same ports — the setup script only manages its own project; containers from other projects must be stopped manually (`docker ps` to find them, `docker stop` to kill them)
@@ -287,7 +191,7 @@ Standalone runs without authentication (`FEATURE_REQUIRE_LOGIN=false`, `AUTH_BAC
1. In `www/.env.local`: set `FEATURE_REQUIRE_LOGIN=true`, uncomment `AUTHENTIK_ISSUER` and `AUTHENTIK_REFRESH_TOKEN_URL`
2. In `server/.env`: set `AUTH_BACKEND=authentik` (or your backend), configure `AUTH_JWT_AUDIENCE`
3. Restart: `docker compose -f docker-compose.standalone.yml up -d --force-recreate web server`
3. Restart: `docker compose -f docker-compose.yml -f docker-compose.standalone.yml up -d --force-recreate web server`
## What's NOT covered

View File

@@ -1,472 +0,0 @@
# How the Self-Hosted Setup Works
This document explains the internals of the self-hosted deployment: how the setup script orchestrates everything, how the Docker Compose profiles work, how services communicate, and how configuration flows from flags to running containers.
> For quick-start instructions and flag reference, see [Self-Hosted Production Deployment](selfhosted-production.md).
## Table of Contents
- [Overview](#overview)
- [The Setup Script Step by Step](#the-setup-script-step-by-step)
- [Docker Compose Profile System](#docker-compose-profile-system)
- [Service Architecture](#service-architecture)
- [Configuration Flow](#configuration-flow)
- [Storage Architecture](#storage-architecture)
- [SSL/TLS and Reverse Proxy](#ssltls-and-reverse-proxy)
- [Build vs Pull Workflow](#build-vs-pull-workflow)
- [Background Task Processing](#background-task-processing)
- [Network and Port Layout](#network-and-port-layout)
---
## Overview
The self-hosted deployment runs the entire Reflector platform on a single server using Docker Compose. A single bash script (`scripts/setup-selfhosted.sh`) handles all configuration and orchestration. The key design principles are:
- **One command to deploy** — flags select which features to enable
- **Idempotent** — safe to re-run without losing existing configuration
- **Profile-based composition** — Docker Compose profiles activate optional services
- **No external dependencies required** — with `--garage` and `--ollama-*`, everything runs locally
## The Setup Script Step by Step
The script (`scripts/setup-selfhosted.sh`) runs 7 sequential steps. Here's what each one does and why.
### Step 0: Prerequisites
Validates the environment before doing anything:
- **Docker Compose V2** — checks `docker compose version` output (not the legacy `docker-compose`)
- **Docker daemon** — verifies `docker info` succeeds
- **NVIDIA GPU** — only checked when `--gpu` or `--ollama-gpu` is used; runs `nvidia-smi` to confirm drivers are installed
- **Compose file** — verifies `docker-compose.selfhosted.yml` exists at the expected path
If any check fails, the script exits with a clear error message and remediation steps.
### Step 1: Generate Secrets
Creates cryptographic secrets needed by the backend and frontend:
- **`SECRET_KEY`** — used by the FastAPI server for session signing (64 hex chars via `openssl rand -hex 32`)
- **`NEXTAUTH_SECRET`** — used by Next.js NextAuth for JWT signing
Secrets are only generated if they don't already exist or are still set to the placeholder value `changeme`. This is what makes the script idempotent for secrets.
If `--password` is passed, this step also generates a PBKDF2-SHA256 password hash from the provided password. The hash is computed using Python's stdlib (`hashlib.pbkdf2_hmac`) with 100,000 iterations and a random 16-byte salt, producing a hash in the format `pbkdf2:sha256:100000$<salt_hex>$<hash_hex>`.
### Step 2: Generate `server/.env`
Creates or updates the backend environment file from `server/.env.selfhosted.example`. Sets:
- **Infrastructure** — PostgreSQL URL, Redis host, Celery broker (all pointing to Docker-internal hostnames)
- **Public URLs** — `BASE_URL` and `CORS_ORIGIN` computed from the domain (if `--domain`), IP (if detected on Linux), or `localhost`
- **WebRTC** — `WEBRTC_HOST` set to the server's LAN IP so browsers can reach UDP ICE candidates
- **Specialized models** — always points to `http://transcription:8000` (the Docker network alias shared by GPU and CPU containers)
- **HuggingFace token** — prompts interactively for pyannote model access; writes to root `.env` so Docker Compose can inject it into GPU/CPU containers
- **LLM** — if `--ollama-*` is used, configures `LLM_URL` pointing to the Ollama container. Otherwise, warns that the user needs to configure an external LLM
- **Public mode** — sets `PUBLIC_MODE=true` so the app is accessible without authentication by default
- **Password auth** — if `--password` is passed, sets `AUTH_BACKEND=password`, `PUBLIC_MODE=false`, `ADMIN_EMAIL=admin@localhost`, and `ADMIN_PASSWORD_HASH` (the hash generated in Step 1). The admin user is provisioned in the database on container startup via `runserver.sh`
The script uses `env_set` for each variable, which either updates an existing line or appends a new one. This means re-running the script updates values in-place without duplicating keys.
### Step 3: Generate `www/.env`
Creates or updates the frontend environment file from `www/.env.selfhosted.example`. Sets:
- **`SITE_URL` / `NEXTAUTH_URL` / `API_URL`** — all set to the same public-facing URL (with `https://` if Caddy is enabled)
- **`WEBSOCKET_URL`** — set to `auto`, which tells the frontend to derive the WebSocket URL from the page URL automatically
- **`SERVER_API_URL`** — always `http://server:1250` (Docker-internal, used for server-side rendering)
- **`KV_URL`** — Redis URL for Next.js caching
- **`FEATURE_REQUIRE_LOGIN`** — `false` by default (matches `PUBLIC_MODE=true` on the backend)
- **Password auth** — if `--password` is passed, sets `FEATURE_REQUIRE_LOGIN=true` and `AUTH_PROVIDER=credentials`, which tells the frontend to use a local email/password login form instead of Authentik OAuth
### Step 4: Storage Setup
Branches based on whether `--garage` was passed:
**With `--garage` (local S3):**
1. Generates `data/garage.toml` from a template, injecting a random RPC secret
2. Starts only the Garage container (`docker compose --profile garage up -d garage`)
3. Waits for the Garage admin API to respond on port 3903
4. Assigns the node to a storage layout (1GB capacity, zone `dc1`)
5. Creates the `reflector-media` bucket
6. Creates an access key named `reflector` and grants it read/write on the bucket
7. Writes all S3 credentials (`ENDPOINT_URL`, `BUCKET_NAME`, `REGION`, `ACCESS_KEY_ID`, `SECRET_ACCESS_KEY`) to `server/.env`
The Garage endpoint is `http://garage:3900` (Docker-internal), and the region is set to `garage` (arbitrary, Garage ignores it). The boto3 client uses path-style addressing when an endpoint URL is configured, which is required for S3-compatible services like Garage.
**Without `--garage` (external S3):**
1. Checks `server/.env` for the four required S3 variables
2. If any are missing, prompts interactively for each one
3. Optionally prompts for an endpoint URL (for MinIO, Backblaze B2, etc.)
### Step 5: Caddyfile
Only runs when `--caddy` or `--domain` is used. Generates a Caddy configuration file:
**With `--domain`:** Creates a named site block (`reflector.example.com { ... }`). Caddy automatically provisions a Let's Encrypt certificate for this domain. Requires DNS pointing to the server and ports 80/443 open.
**Without `--domain` (IP access):** Creates a catch-all `:443 { tls internal ... }` block. Caddy generates a self-signed certificate. Browsers will show a security warning.
Both configurations route:
- `/v1/*` and `/health` to the backend (`server:1250`)
- Everything else to the frontend (`web:3000`)
### Step 6: Start Services
1. **Always builds the GPU/CPU model image** — these are never prebuilt because they contain ML model download logic specific to the host's hardware
2. **With `--build`:** Also builds backend (server, worker, beat) and frontend (web) images from source
3. **Without `--build`:** Pulls prebuilt images from the Docker registry (`monadicalsas/reflector-backend:latest`, `monadicalsas/reflector-frontend:latest`)
4. **Starts all services**`docker compose up -d` with the active profiles
5. **Quick sanity check** — after 3 seconds, checks for any containers that exited immediately
### Step 7: Health Checks
Waits for each service in order, with generous timeouts:
| Service | Check | Timeout | Notes |
|---------|-------|---------|-------|
| GPU/CPU models | `curl http://localhost:8000/docs` | 10 min (120 x 5s) | First start downloads ~1GB of models |
| Ollama | `curl http://localhost:11435/api/tags` | 3 min (60 x 3s) | Then pulls the selected model |
| Server API | `curl http://localhost:1250/health` | 7.5 min (90 x 5s) | First start runs database migrations |
| Frontend | `curl http://localhost:3000` | 1.5 min (30 x 3s) | Next.js build on first start |
| Caddy | `curl -k https://localhost` | Quick check | After other services are up |
If the server container exits during the health check, the script dumps diagnostics (container statuses + logs) before exiting.
After the Ollama health check passes, the script checks if the selected model is already pulled. If not, it runs `ollama pull <model>` inside the container.
---
## Docker Compose Profile System
The compose file (`docker-compose.selfhosted.yml`) uses Docker Compose profiles to make services optional. Only services whose profiles match the active `--profile` flags are started.
### Always-on Services (no profile)
These start regardless of which flags you pass:
| Service | Role | Image |
|---------|------|-------|
| `server` | FastAPI backend, API endpoints, WebRTC | `monadicalsas/reflector-backend:latest` |
| `worker` | Celery worker for background processing | Same image, `ENTRYPOINT=worker` |
| `beat` | Celery beat scheduler for periodic tasks | Same image, `ENTRYPOINT=beat` |
| `web` | Next.js frontend | `monadicalsas/reflector-frontend:latest` |
| `redis` | Message broker + caching | `redis:7.2-alpine` |
| `postgres` | Primary database | `postgres:17-alpine` |
### Profile-Based Services
| Profile | Service | Role |
|---------|---------|------|
| `gpu` | `gpu` | NVIDIA GPU-accelerated transcription/diarization/translation |
| `cpu` | `cpu` | CPU-only transcription/diarization/translation |
| `ollama-gpu` | `ollama` | Local Ollama LLM with GPU |
| `ollama-cpu` | `ollama-cpu` | Local Ollama LLM on CPU |
| `garage` | `garage` | Local S3-compatible object storage |
| `caddy` | `caddy` | Reverse proxy with SSL |
### The "transcription" Alias
Both the `gpu` and `cpu` services define a Docker network alias of `transcription`. This means the backend always connects to `http://transcription:8000` regardless of which profile is active. The alias is defined in the compose file's `networks.default.aliases` section.
---
## Service Architecture
```
┌─────────────┐
Internet ────────>│ Caddy │ :80/:443 (profile: caddy)
└──────┬──────┘
┌────────────┼────────────┐
│ │ │
v v │
┌─────────┐ ┌─────────┐ │
│ web │ │ server │ │
│ :3000 │ │ :1250 │ │
└─────────┘ └────┬────┘ │
│ │
┌────┴────┐ │
│ worker │ │
│ beat │ │
└────┬────┘ │
│ │
┌──────────────┼────────────┤
│ │ │
v v v
┌───────────┐ ┌─────────┐ ┌─────────┐
│transcription│ │postgres │ │ redis │
│ (gpu/cpu) │ │ :5432 │ │ :6379 │
│ :8000 │ └─────────┘ └─────────┘
└───────────┘
┌─────┴─────┐ ┌─────────┐
│ ollama │ │ garage │
│(optional) │ │(optional│
│ :11435 │ │ S3) │
└───────────┘ └─────────┘
```
### How Services Interact
1. **User request** hits Caddy (if enabled), which routes to `web` (pages) or `server` (API)
2. **`web`** renders pages server-side using `SERVER_API_URL=http://server:1250` and client-side using the public `API_URL`
3. **`server`** handles API requests, file uploads, WebRTC streaming. Dispatches background work to Celery via Redis
4. **`worker`** picks up Celery tasks (transcription pipelines, audio processing). Calls `transcription:8000` for ML inference and uploads results to S3 storage
5. **`beat`** schedules periodic tasks (cleanup, webhook retries) by pushing them onto the Celery queue
6. **`transcription` (gpu/cpu)** runs Whisper/Parakeet (transcription), Pyannote (diarization), and translation models. Stateless HTTP API
7. **`ollama`** provides an OpenAI-compatible API for summarization and topic detection. Called by the worker during post-processing
8. **`garage`** provides S3-compatible storage for audio files and processed results. Accessed by the worker via boto3
---
## Configuration Flow
Environment variables flow through multiple layers. Understanding this prevents confusion when debugging:
```
Flags (--gpu, --garage, etc.)
├── setup-selfhosted.sh interprets flags
│ │
│ ├── Writes server/.env (backend config)
│ ├── Writes www/.env (frontend config)
│ ├── Writes .env (HF_TOKEN for compose interpolation)
│ └── Writes Caddyfile (proxy routes)
└── docker-compose.selfhosted.yml reads:
├── env_file: ./server/.env (loaded into server, worker, beat)
├── env_file: ./www/.env (loaded into web)
├── .env (compose variable interpolation, e.g. ${HF_TOKEN})
└── environment: {...} (hardcoded overrides, always win over env_file)
```
### Precedence Rules
Docker Compose `environment:` keys **always override** `env_file:` values. This is by design — the compose file hardcodes infrastructure values that must be correct inside the Docker network (like `DATABASE_URL=postgresql+asyncpg://...@postgres:5432/...`) regardless of what's in `server/.env`.
The `server/.env` file is still useful for:
- Values not overridden in the compose file (LLM config, storage credentials, auth settings)
- Running the server outside Docker during development
### The Three `.env` Files
| File | Used By | Contains |
|------|---------|----------|
| `server/.env` | server, worker, beat | Backend config: database, Redis, S3, LLM, auth, public URLs |
| `www/.env` | web | Frontend config: site URL, auth, feature flags |
| `.env` (root) | Docker Compose interpolation | Only `HF_TOKEN` — injected into GPU/CPU container env |
---
## Storage Architecture
All audio files and processing results are stored in S3-compatible object storage. The backend uses boto3 (via aioboto3) with automatic path-style addressing when a custom endpoint URL is configured.
### How Garage Works
Garage is a lightweight, self-hosted S3-compatible storage engine. In this deployment:
- Runs as a single-node cluster with 1GB capacity allocation
- Listens on port 3900 (S3 API) and 3903 (admin API)
- Data persists in Docker volumes (`garage_data`, `garage_meta`)
- Accessed by the worker at `http://garage:3900` (Docker-internal)
The setup script creates:
- A bucket called `reflector-media`
- An access key called `reflector` with read/write permissions on that bucket
### Path-Style vs Virtual-Hosted Addressing
AWS S3 uses virtual-hosted addressing by default (`bucket.s3.amazonaws.com`). S3-compatible services like Garage require path-style addressing (`endpoint/bucket`). The `AwsStorage` class detects this automatically: when `TRANSCRIPT_STORAGE_AWS_ENDPOINT_URL` is set, it configures boto3 with `addressing_style: "path"`.
---
## SSL/TLS and Reverse Proxy
### With `--domain` (Production)
Caddy automatically obtains and renews a Let's Encrypt certificate. Requirements:
- DNS A record pointing to the server
- Ports 80 (HTTP challenge) and 443 (HTTPS) open to the internet
The generated Caddyfile uses the domain as the site address, which triggers Caddy's automatic HTTPS.
### Without `--domain` (Development/LAN)
Caddy generates a self-signed certificate and listens on `:443` as a catch-all. Browsers will show a security warning that must be accepted manually.
### Without `--caddy` (BYO Proxy)
No ports are exposed to the internet. The services listen on `127.0.0.1` only:
- Frontend: `localhost:3000`
- Backend API: `localhost:1250`
You can point your own reverse proxy (nginx, Traefik, etc.) at these ports.
### WebRTC and UDP
The server exposes UDP ports 50000-50100 for WebRTC ICE candidates. The `WEBRTC_HOST` variable tells the server which IP to advertise in ICE candidates — this must be the server's actual IP address (not a domain), because WebRTC uses UDP which doesn't go through the HTTP reverse proxy.
---
## Build vs Pull Workflow
### Default (no `--build` flag)
```
GPU/CPU model image: Always built from source (./gpu/self_hosted/)
Backend image: Pulled from monadicalsas/reflector-backend:latest
Frontend image: Pulled from monadicalsas/reflector-frontend:latest
```
The GPU/CPU image is always built because it contains hardware-specific build steps and ML model download logic.
### With `--build`
```
GPU/CPU model image: Built from source (./gpu/self_hosted/)
Backend image: Built from source (./server/)
Frontend image: Built from source (./www/)
```
Use `--build` when:
- You've made local code changes
- The prebuilt registry images are outdated
- You want to verify the build works on your hardware
### Rebuilding Individual Services
```bash
# Rebuild just the backend
docker compose -f docker-compose.selfhosted.yml build server worker beat
# Rebuild just the frontend
docker compose -f docker-compose.selfhosted.yml build web
# Rebuild the GPU model container
docker compose -f docker-compose.selfhosted.yml build gpu
# Force a clean rebuild (no cache)
docker compose -f docker-compose.selfhosted.yml build --no-cache server
```
---
## Background Task Processing
### Celery Architecture
The backend uses Celery for all background work, with Redis as the message broker:
- **`worker`** — picks up tasks from the Redis queue and executes them
- **`beat`** — schedules periodic tasks (cron-like) by pushing them onto the queue
- **`Redis`** — acts as both message broker and result backend
### The Audio Processing Pipeline
When a file is uploaded, the worker runs a multi-step pipeline:
```
Upload → Extract Audio → Upload to S3
┌──────┼──────┐
│ │ │
v v v
Transcribe Diarize Waveform
│ │ │
└──────┼──────┘
Assemble
┌──────┼──────┐
v v v
Topics Title Summaries
Done
```
Transcription, diarization, and waveform generation run in parallel. After assembly, topic detection, title generation, and summarization also run in parallel. Each step calls the appropriate service (transcription container for ML, Ollama/external LLM for text generation, S3 for storage).
### Event Loop Management
Each Celery task runs in its own `asyncio.run()` call, which creates a fresh event loop. The `asynctask` decorator in `server/reflector/asynctask.py` handles:
1. **Database connections** — resets the connection pool before each task (connections from a previous event loop would cause "Future attached to a different loop" errors)
2. **Redis connections** — resets the WebSocket manager singleton so Redis pub/sub reconnects on the current loop
3. **Cleanup** — disconnects the database and clears the context variable in the `finally` block
---
## Network and Port Layout
All services communicate over Docker's default bridge network. Only specific ports are exposed to the host:
| Port | Service | Binding | Purpose |
|------|---------|---------|---------|
| 80 | Caddy | `0.0.0.0:80` | HTTP (redirect to HTTPS / Let's Encrypt challenge) |
| 443 | Caddy | `0.0.0.0:443` | HTTPS (main entry point) |
| 1250 | Server | `127.0.0.1:1250` | Backend API (localhost only) |
| 3000 | Web | `127.0.0.1:3000` | Frontend (localhost only) |
| 3900 | Garage | `0.0.0.0:3900` | S3 API (for admin/debug access) |
| 3903 | Garage | `0.0.0.0:3903` | Garage admin API |
| 8000 | GPU/CPU | `127.0.0.1:8000` | ML model API (localhost only) |
| 11435 | Ollama | `127.0.0.1:11435` | Ollama API (localhost only) |
| 50000-50100/udp | Server | `0.0.0.0:50000-50100` | WebRTC ICE candidates |
Services bound to `127.0.0.1` are only accessible from the host itself (not from the network). Caddy is the only service exposed to the internet on standard HTTP/HTTPS ports.
### Docker-Internal Hostnames
Inside the Docker network, services reach each other by their compose service name:
| Hostname | Resolves To |
|----------|-------------|
| `server` | Backend API container |
| `web` | Frontend container |
| `postgres` | PostgreSQL container |
| `redis` | Redis container |
| `transcription` | GPU or CPU container (network alias) |
| `ollama` / `ollama-cpu` | Ollama container |
| `garage` | Garage S3 container |
---
## Diagnostics and Error Handling
The setup script includes an `ERR` trap that automatically dumps diagnostics when any command fails:
1. Lists all container statuses
2. Shows the last 30 lines of logs for any stopped/exited containers
3. Shows the last 40 lines of the specific failing service
This means if something goes wrong during setup, you'll see the relevant logs immediately without having to run manual debug commands.
### Common Debug Commands
```bash
# Overall status
docker compose -f docker-compose.selfhosted.yml ps
# Logs for a specific service
docker compose -f docker-compose.selfhosted.yml logs server --tail 50
docker compose -f docker-compose.selfhosted.yml logs worker --tail 50
# Check environment inside a container
docker compose -f docker-compose.selfhosted.yml exec server env | grep TRANSCRIPT
# Health check from inside the network
docker compose -f docker-compose.selfhosted.yml exec server curl http://localhost:1250/health
# Check S3 storage connectivity
docker compose -f docker-compose.selfhosted.yml exec server curl http://garage:3900
# Database access
docker compose -f docker-compose.selfhosted.yml exec postgres psql -U reflector -c "SELECT id, status FROM transcript ORDER BY created_at DESC LIMIT 5;"
# List files in server data directory
docker compose -f docker-compose.selfhosted.yml exec server ls -la /app/data/
```

View File

@@ -1,522 +0,0 @@
# Self-Hosted Production Deployment
Deploy Reflector on a single server with everything running in Docker. Transcription, diarization, and translation use specialized ML models (Whisper/Parakeet, Pyannote); only summarization and topic detection require an LLM.
> For a detailed walkthrough of how the setup script and infrastructure work under the hood, see [How the Self-Hosted Setup Works](selfhosted-architecture.md).
## Prerequisites
### Hardware
- **With GPU**: Linux server with NVIDIA GPU (8GB+ VRAM recommended), 16GB+ RAM, 50GB+ disk
- **CPU-only**: 8+ cores, 32GB+ RAM (transcription is slower but works)
- Disk space for ML models (~2GB on first run) + audio storage
### Software
- Docker Engine 24+ with Compose V2
- NVIDIA drivers + `nvidia-container-toolkit` (GPU modes only)
- `curl`, `openssl` (usually pre-installed)
### Accounts & Credentials (depending on options)
**Always recommended:**
- **HuggingFace token** — For downloading pyannote speaker diarization models. Get one at https://huggingface.co/settings/tokens and accept the model licenses:
- https://huggingface.co/pyannote/speaker-diarization-3.1
- https://huggingface.co/pyannote/segmentation-3.0
- The setup script will prompt for this. If skipped, diarization falls back to a public model bundle (may be less reliable).
**LLM for summarization & topic detection (pick one):**
- **With `--ollama-gpu` or `--ollama-cpu`**: Nothing extra — Ollama runs locally and pulls the model automatically
- **Without `--ollama-*`**: An OpenAI-compatible LLM API key and endpoint. Examples:
- OpenAI: `LLM_URL=https://api.openai.com/v1`, `LLM_API_KEY=sk-...`, `LLM_MODEL=gpt-4o-mini`
- Anthropic, Together, Groq, or any OpenAI-compatible API
- A self-managed vLLM or Ollama instance elsewhere on the network
**Object storage (pick one):**
- **With `--garage`**: Nothing extra — Garage (local S3-compatible storage) is auto-configured by the script
- **Without `--garage`**: S3-compatible storage credentials. The script will prompt for these, or you can pre-fill `server/.env`. Options include:
- **AWS S3**: Access Key ID, Secret Access Key, bucket name, region
- **MinIO**: Same credentials + `TRANSCRIPT_STORAGE_AWS_ENDPOINT_URL=http://your-minio:9000`
- **Any S3-compatible provider** (Backblaze B2, Cloudflare R2, DigitalOcean Spaces, etc.): same fields + custom endpoint URL
**Optional add-ons (configure after initial setup):**
- **Authentik** (user authentication): Requires an Authentik instance with an OAuth2/OIDC application configured for Reflector. See [Enabling Authentication](#enabling-authentication-authentik) below.
## Quick Start
```bash
git clone https://github.com/Monadical-SAS/reflector.git
cd reflector
# GPU + local Ollama LLM + local Garage storage + Caddy SSL (with domain):
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --garage --caddy --domain reflector.example.com
# Same but without a domain (self-signed cert, access via IP):
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --garage --caddy
# CPU-only (same, but slower):
./scripts/setup-selfhosted.sh --cpu --ollama-cpu --garage --caddy
# With password authentication (single admin user):
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --garage --caddy --password mysecretpass
# Build from source instead of pulling prebuilt images:
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --garage --caddy --build
```
That's it. The script generates env files, secrets, starts all containers, waits for health checks, and prints the URL.
## Specialized Models (Required)
Pick `--gpu` or `--cpu`. This determines how **transcription, diarization, and translation** run:
| Flag | What it does | Requires |
|------|-------------|----------|
| `--gpu` | NVIDIA GPU acceleration for ML models | NVIDIA GPU + drivers + `nvidia-container-toolkit` |
| `--cpu` | CPU-only (slower but works without GPU) | 8+ cores, 32GB+ RAM recommended |
## Local LLM (Optional)
Optionally add `--ollama-gpu` or `--ollama-cpu` for a **local Ollama instance** that handles summarization and topic detection. If omitted, configure an external OpenAI-compatible LLM in `server/.env`.
| Flag | What it does | Requires |
|------|-------------|----------|
| `--ollama-gpu` | Local Ollama with NVIDIA GPU acceleration | NVIDIA GPU |
| `--ollama-cpu` | Local Ollama on CPU only | Nothing extra |
| `--llm-model MODEL` | Choose which Ollama model to download (default: `qwen2.5:14b`) | `--ollama-gpu` or `--ollama-cpu` |
| *(omitted)* | User configures external LLM (OpenAI, Anthropic, etc.) | LLM API key |
### macOS / Apple Silicon
`--ollama-gpu` requires an NVIDIA GPU and **does not work on macOS**. Docker on macOS cannot access Apple GPU acceleration, so the containerized Ollama will run on CPU only regardless of the flag used.
For the best performance on Mac, we recommend running Ollama **natively outside Docker** (install from https://ollama.com) — this gives Ollama direct access to Apple Metal GPU acceleration. Then omit `--ollama-gpu`/`--ollama-cpu` from the setup script and point the backend to your local Ollama instance:
```env
# In server/.env
LLM_URL=http://host.docker.internal:11434/v1
LLM_MODEL=qwen2.5:14b
LLM_API_KEY=not-needed
```
`--ollama-cpu` does work on macOS but will be significantly slower than a native Ollama install with Metal acceleration.
### Choosing an Ollama model
The default model is `qwen2.5:14b` (~9GB download, good multilingual support and summary quality). Override with `--llm-model`:
```bash
# Default (qwen2.5:14b)
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --garage --caddy
# Mistral — good balance of speed and quality (~4.1GB)
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --llm-model mistral --garage --caddy
# Phi-4 — smaller and faster (~9.1GB)
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --llm-model phi4 --garage --caddy
# Llama 3.3 70B — best quality, needs 48GB+ RAM or GPU VRAM (~43GB)
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --llm-model llama3.3:70b --garage --caddy
# Gemma 2 9B (~5.4GB)
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --llm-model gemma2 --garage --caddy
# DeepSeek R1 8B — reasoning model, verbose but thorough summaries (~4.9GB)
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --llm-model deepseek-r1:8b --garage --caddy
```
Browse all available models at https://ollama.com/library.
### Recommended combinations
- **`--gpu --ollama-gpu`**: Best for servers with NVIDIA GPU. Fully self-contained, no external API keys needed.
- **`--cpu --ollama-cpu`**: No GPU available but want everything self-contained. Slower but works.
- **`--gpu --ollama-cpu`**: GPU for transcription, CPU for LLM. Saves GPU VRAM for ML models.
- **`--gpu`**: Have NVIDIA GPU but prefer a cloud LLM (faster/better summaries with GPT-4, Claude, etc.).
- **`--cpu`**: No GPU, prefer cloud LLM. Slowest transcription but best summary quality.
## Other Optional Flags
| Flag | What it does |
|------|-------------|
| `--garage` | Starts Garage (local S3-compatible storage). Auto-configures bucket, keys, and env vars. |
| `--caddy` | Starts Caddy reverse proxy on ports 80/443 with self-signed cert. |
| `--domain DOMAIN` | Use a real domain with Let's Encrypt auto-HTTPS (implies `--caddy`). Requires DNS A record pointing to this server and ports 80/443 open. |
| `--password PASS` | Enable password authentication with an `admin@localhost` user. Sets `AUTH_BACKEND=password`, `PUBLIC_MODE=false`. See [Enabling Password Authentication](#enabling-password-authentication). |
| `--build` | Build backend (server, worker, beat) and frontend (web) Docker images from source instead of pulling prebuilt images from the registry. Useful for development or when running a version with local changes. |
Without `--garage`, you **must** provide S3-compatible credentials (the script will prompt interactively or you can pre-fill `server/.env`).
Without `--caddy` or `--domain`, no ports are exposed. Point your own reverse proxy at `web:3000` (frontend) and `server:1250` (API).
**Using a domain (recommended for production):** Point a DNS A record at your server's IP, then pass `--domain your.domain.com`. Caddy will automatically obtain and renew a Let's Encrypt certificate. Ports 80 and 443 must be open.
**Without a domain:** `--caddy` alone uses a self-signed certificate. Browsers will show a security warning that must be accepted.
## What the Script Does
1. **Prerequisites check** — Docker, NVIDIA GPU (if needed), compose file exists
2. **Generate secrets**`SECRET_KEY`, `NEXTAUTH_SECRET` via `openssl rand`
3. **Generate `server/.env`** — From template, sets infrastructure defaults, configures LLM based on mode, enables `PUBLIC_MODE`
4. **Generate `www/.env`** — Auto-detects server IP, sets URLs
5. **Storage setup** — Either initializes Garage (bucket, keys, permissions) or prompts for external S3 credentials
6. **Caddyfile** — Generates domain-specific (Let's Encrypt) or IP-specific (self-signed) configuration
7. **Build & start** — Always builds GPU/CPU model image from source. With `--build`, also builds backend and frontend from source; otherwise pulls prebuilt images from the registry
8. **Health checks** — Waits for each service, pulls Ollama model if needed, warns about missing LLM config
> For a deeper dive into each step, see [How the Self-Hosted Setup Works](selfhosted-architecture.md).
## Configuration Reference
### Server Environment (`server/.env`)
| Variable | Description | Default |
|----------|-------------|---------|
| `DATABASE_URL` | PostgreSQL connection | Auto-set (Docker internal) |
| `REDIS_HOST` | Redis hostname | Auto-set (`redis`) |
| `SECRET_KEY` | App secret | Auto-generated |
| `AUTH_BACKEND` | Authentication method (`none`, `password`, `jwt`) | `none` |
| `PUBLIC_MODE` | Allow unauthenticated access | `true` |
| `ADMIN_EMAIL` | Admin email for password auth | *(unset)* |
| `ADMIN_PASSWORD_HASH` | PBKDF2 hash for password auth | *(unset)* |
| `WEBRTC_HOST` | IP advertised in WebRTC ICE candidates | Auto-detected (server IP) |
| `TRANSCRIPT_URL` | Specialized model endpoint | `http://transcription:8000` |
| `LLM_URL` | OpenAI-compatible LLM endpoint | Auto-set for Ollama modes |
| `LLM_API_KEY` | LLM API key | `not-needed` for Ollama |
| `LLM_MODEL` | LLM model name | `qwen2.5:14b` for Ollama (override with `--llm-model`) |
| `CELERY_BEAT_POLL_INTERVAL` | Override all worker polling intervals (seconds). `0` = use individual defaults | `300` (selfhosted), `0` (other) |
| `TRANSCRIPT_STORAGE_BACKEND` | Storage backend | `aws` |
| `TRANSCRIPT_STORAGE_AWS_*` | S3 credentials | Auto-set for Garage |
### Frontend Environment (`www/.env`)
| Variable | Description | Default |
|----------|-------------|---------|
| `SITE_URL` | Public-facing URL | Auto-detected |
| `API_URL` | API URL (browser-side) | Same as SITE_URL |
| `SERVER_API_URL` | API URL (server-side) | `http://server:1250` |
| `NEXTAUTH_SECRET` | Auth secret | Auto-generated |
| `FEATURE_REQUIRE_LOGIN` | Require authentication | `false` |
| `AUTH_PROVIDER` | Auth provider (`authentik` or `credentials`) | *(unset)* |
## Storage Options
### Garage (Recommended for Self-Hosted)
Use `--garage` flag. The script automatically:
- Generates `data/garage.toml` with a random RPC secret
- Starts the Garage container
- Creates the `reflector-media` bucket
- Creates an access key with read/write permissions
- Writes all S3 credentials to `server/.env`
### External S3 (AWS, MinIO, etc.)
Don't use `--garage`. The script will prompt for:
- Access Key ID
- Secret Access Key
- Bucket Name
- Region
- Endpoint URL (for non-AWS like MinIO)
Or pre-fill in `server/.env`:
```env
TRANSCRIPT_STORAGE_BACKEND=aws
TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID=your-key
TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY=your-secret
TRANSCRIPT_STORAGE_AWS_BUCKET_NAME=reflector-media
TRANSCRIPT_STORAGE_AWS_REGION=us-east-1
# For non-AWS S3 (MinIO, etc.):
TRANSCRIPT_STORAGE_AWS_ENDPOINT_URL=http://minio:9000
```
## What Authentication Enables
By default, Reflector runs in **public mode** (`AUTH_BACKEND=none`, `PUBLIC_MODE=true`) — anyone can create and view transcripts without logging in. Transcripts are anonymous (not linked to any user) and cannot be edited or deleted after creation.
Enabling authentication (either password or Authentik) unlocks:
| Feature | Public mode (no auth) | With authentication |
|---------|----------------------|---------------------|
| Create transcripts (record/upload) | Yes (anonymous, unowned) | Yes (owned by user) |
| View transcripts | All transcripts visible | Own transcripts + shared rooms |
| Edit/delete transcripts | No | Yes (owner only) |
| Privacy controls (private/semi-private/public) | No (everything public) | Yes (owner can set share mode) |
| Speaker reassignment and merging | No | Yes (owner only) |
| Participant management (add/edit/delete) | Read-only | Full CRUD (owner only) |
| Create rooms | No | Yes |
| Edit/delete rooms | No | Yes (owner only) |
| Room calendar (ICS) sync | No | Yes (owner only) |
| API key management | No | Yes |
| Post to Zulip | No | Yes (owner only) |
| Real-time WebSocket notifications | No (connection closed) | Yes (transcript create/delete events) |
| Meeting host access (Daily.co token) | No | Yes (room owner) |
In short: public mode is "demo-friendly" — great for trying Reflector out. Authentication adds **ownership, privacy, and management** of your data.
## Authentication Options
Reflector supports three authentication backends:
| Backend | `AUTH_BACKEND` | Use case |
|---------|---------------|----------|
| `none` | `none` | Public/demo mode, no login required |
| `password` | `password` | Single-user self-hosted, simple email/password login |
| `jwt` | `jwt` | Multi-user via Authentik (OAuth2/OIDC) |
## Enabling Password Authentication
The simplest way to add authentication. Creates a single admin user with email/password login — no external identity provider needed.
### Quick setup (recommended)
Pass `--password` to the setup script:
```bash
./scripts/setup-selfhosted.sh --gpu --ollama-gpu --garage --caddy --password mysecretpass
```
This automatically:
- Sets `AUTH_BACKEND=password` and `PUBLIC_MODE=false` in `server/.env`
- Creates an `admin@localhost` user with the given password
- Sets `FEATURE_REQUIRE_LOGIN=true` and `AUTH_PROVIDER=credentials` in `www/.env`
- Provisions the admin user in the database on container startup
### Manual setup
If you prefer to configure manually or want to change the admin email:
1. Generate a password hash:
```bash
cd server
uv run python -m reflector.tools.create_admin --hash-only --password yourpassword
```
2. Update `server/.env`:
```env
AUTH_BACKEND=password
PUBLIC_MODE=false
ADMIN_EMAIL=admin@yourdomain.com
ADMIN_PASSWORD_HASH=pbkdf2:sha256:100000$<salt>$<hash>
```
3. Update `www/.env`:
```env
FEATURE_REQUIRE_LOGIN=true
AUTH_PROVIDER=credentials
```
4. Restart:
```bash
docker compose -f docker-compose.selfhosted.yml down
./scripts/setup-selfhosted.sh <same-flags>
```
### How it works
- The backend issues HS256 JWTs (signed with `SECRET_KEY`) on successful login via `POST /v1/auth/login`
- Tokens expire after 24 hours; the user must log in again after expiry
- The frontend shows a login page at `/login` with email and password fields
- A rate limiter blocks IPs after 10 failed login attempts within 5 minutes
- The admin user is provisioned automatically on container startup from `ADMIN_EMAIL` and `ADMIN_PASSWORD_HASH` environment variables
- Passwords are hashed with PBKDF2-SHA256 (100,000 iterations) — no additional dependencies required
### Changing the admin password
```bash
cd server
uv run python -m reflector.tools.create_admin --email admin@localhost --password newpassword
```
Or update `ADMIN_PASSWORD_HASH` in `server/.env` and restart the containers.
## Enabling Authentication (Authentik)
For multi-user deployments with SSO. Requires an external Authentik instance.
By default, authentication is disabled (`AUTH_BACKEND=none`, `FEATURE_REQUIRE_LOGIN=false`). To enable:
1. Deploy an Authentik instance (see [Authentik docs](https://goauthentik.io/docs/installation))
2. Create an OAuth2/OIDC application for Reflector
3. Update `server/.env`:
```env
AUTH_BACKEND=jwt
AUTH_JWT_AUDIENCE=your-client-id
```
4. Update `www/.env`:
```env
FEATURE_REQUIRE_LOGIN=true
AUTH_PROVIDER=authentik
AUTHENTIK_ISSUER=https://authentik.example.com/application/o/reflector
AUTHENTIK_REFRESH_TOKEN_URL=https://authentik.example.com/application/o/token/
AUTHENTIK_CLIENT_ID=your-client-id
AUTHENTIK_CLIENT_SECRET=your-client-secret
```
5. Restart: `docker compose -f docker-compose.selfhosted.yml down && ./scripts/setup-selfhosted.sh <same-flags>`
## Enabling Real Domain with Let's Encrypt
By default, Caddy uses self-signed certificates. For a real domain:
1. Point your domain's DNS to your server's IP
2. Ensure ports 80 and 443 are open
3. Edit `Caddyfile`:
```
reflector.example.com {
handle /v1/* {
reverse_proxy server:1250
}
handle /health {
reverse_proxy server:1250
}
handle {
reverse_proxy web:3000
}
}
```
4. Update `www/.env`:
```env
SITE_URL=https://reflector.example.com
NEXTAUTH_URL=https://reflector.example.com
API_URL=https://reflector.example.com
```
5. Restart Caddy: `docker compose -f docker-compose.selfhosted.yml restart caddy web`
## Worker Polling Frequency
The selfhosted setup defaults all background worker polling intervals to **300 seconds (5 minutes)** to reduce CPU and memory usage. This controls how often the beat scheduler triggers tasks like recording discovery, meeting reconciliation, and calendar sync.
To change the interval, edit `server/.env`:
```env
# Poll every 60 seconds (more responsive, uses more resources)
CELERY_BEAT_POLL_INTERVAL=60
# Poll every 5 minutes (default for selfhosted)
CELERY_BEAT_POLL_INTERVAL=300
# Use individual per-task defaults (production SaaS behavior)
CELERY_BEAT_POLL_INTERVAL=0
```
After changing, restart the beat and worker containers:
```bash
docker compose -f docker-compose.selfhosted.yml restart beat worker
```
**Affected tasks when `CELERY_BEAT_POLL_INTERVAL` is set:**
| Task | Default (no override) | With override |
|------|-----------------------|---------------|
| SQS message polling | 60s | Override value |
| Daily.co recording discovery | 15s (no webhook) / 180s (webhook) | Override value |
| Meeting reconciliation | 30s | Override value |
| ICS calendar sync | 60s | Override value |
| Upcoming meeting creation | 30s | Override value |
> **Note:** Daily crontab tasks (failed recording reprocessing at 05:00 UTC, public data cleanup at 03:00 UTC) and healthcheck pings (10 min) are **not** affected by this setting.
## Troubleshooting
### Check service status
```bash
docker compose -f docker-compose.selfhosted.yml ps
```
### View logs for a specific service
```bash
docker compose -f docker-compose.selfhosted.yml logs server --tail 50
docker compose -f docker-compose.selfhosted.yml logs gpu --tail 50
docker compose -f docker-compose.selfhosted.yml logs web --tail 50
```
### GPU service taking too long
First start downloads ~1-2GB of ML models. Check progress:
```bash
docker compose -f docker-compose.selfhosted.yml logs gpu -f
```
### Server exits immediately
Usually a database migration issue. Check:
```bash
docker compose -f docker-compose.selfhosted.yml logs server --tail 50
```
### Caddy certificate issues
For self-signed certs, your browser will warn. Click Advanced > Proceed.
For Let's Encrypt, ensure ports 80/443 are open and DNS is pointed correctly.
### Summaries/topics not generating
Check LLM configuration:
```bash
grep LLM_ server/.env
```
If you didn't use `--ollama-gpu` or `--ollama-cpu`, you must set `LLM_URL`, `LLM_API_KEY`, and `LLM_MODEL`.
### Health check from inside containers
```bash
docker compose -f docker-compose.selfhosted.yml exec server curl http://localhost:1250/health
docker compose -f docker-compose.selfhosted.yml exec gpu curl http://localhost:8000/docs
```
## Updating
```bash
# Option A: Pull latest prebuilt images and restart
docker compose -f docker-compose.selfhosted.yml down
./scripts/setup-selfhosted.sh <same-flags-as-before>
# Option B: Build from source (after git pull) and restart
git pull
docker compose -f docker-compose.selfhosted.yml down
./scripts/setup-selfhosted.sh <same-flags-as-before> --build
# Rebuild only the GPU/CPU model image (picks up model updates)
docker compose -f docker-compose.selfhosted.yml build gpu # or cpu
```
The setup script is idempotent — it won't overwrite existing secrets or env vars that are already set.
## Architecture Overview
```
┌─────────┐
Internet ────────>│ Caddy │ :80/:443
└────┬────┘
┌────────────┼────────────┐
│ │ │
v v │
┌─────────┐ ┌─────────┐ │
│ web │ │ server │ │
│ :3000 │ │ :1250 │ │
└─────────┘ └────┬────┘ │
│ │
┌────┴────┐ │
│ worker │ │
│ beat │ │
└────┬────┘ │
│ │
┌──────────────┼────────────┤
│ │ │
v v v
┌───────────┐ ┌─────────┐ ┌─────────┐
│transcription│ │postgres │ │ redis │
│(gpu/cpu) │ │ :5432 │ │ :6379 │
│ :8000 │ └─────────┘ └─────────┘
└───────────┘
┌─────┴─────┐ ┌─────────┐
│ ollama │ │ garage │
│ (optional)│ │(optional│
│ :11435 │ │ S3) │
└───────────┘ └─────────┘
```
All services communicate over Docker's internal network. Only Caddy (if enabled) exposes ports to the internet.
## Future Plans for the Self-Hosted Script
The following features are supported by Reflector but are **not yet integrated into the self-hosted setup script** and require manual configuration:
- **Daily.co live rooms with multitrack processing**: Daily.co enables real-time meeting rooms with automatic recording and per-participant audio tracks for improved diarization. Requires a Daily.co account, API key, and an AWS S3 bucket for recording storage. Currently not automated in the script because the worker orchestration (hatchet) is not yet supported in the selfhosted compose setup.

10
node_modules/.yarn-integrity generated vendored
View File

@@ -1,10 +0,0 @@
{
"systemParams": "darwin-x64-83",
"modulesFolders": [],
"flags": [],
"linkedModules": [],
"topLevelPatterns": [],
"lockfileEntries": {},
"files": [],
"artifacts": {}
}

View File

@@ -1,87 +0,0 @@
#!/usr/bin/env bash
#
# Install Docker Engine + Compose plugin on Ubuntu.
# Ubuntu's default repos don't include docker-compose-plugin, so we add Docker's official repo.
#
# Usage:
# ./scripts/install-docker-ubuntu.sh
#
# Requires: root or sudo
#
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# --- Colors ---
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
CYAN='\033[0;36m'
NC='\033[0m'
info() { echo -e "${CYAN}==>${NC} $*"; }
ok() { echo -e "${GREEN}${NC} $*"; }
warn() { echo -e "${YELLOW} !${NC} $*"; }
err() { echo -e "${RED}${NC} $*" >&2; }
# Use sudo if available and not root; otherwise run directly
if [[ $(id -u) -eq 0 ]]; then
MAYBE_SUDO=""
elif command -v sudo &>/dev/null; then
MAYBE_SUDO="sudo "
else
err "Need root. Run as root or install sudo: apt install sudo"
exit 1
fi
# Check Ubuntu
if [[ ! -f /etc/os-release ]]; then
err "Cannot detect OS. This script is for Ubuntu."
exit 1
fi
source /etc/os-release
if [[ "${ID:-}" != "ubuntu" ]] && [[ "${ID_LIKE:-}" != *"ubuntu"* ]]; then
err "This script is for Ubuntu. Detected: ${ID:-unknown}"
exit 1
fi
info "Adding Docker's official repository..."
${MAYBE_SUDO}apt update
${MAYBE_SUDO}apt install -y ca-certificates curl
${MAYBE_SUDO}install -m 0755 -d /etc/apt/keyrings
${MAYBE_SUDO}rm -f /etc/apt/sources.list.d/docker.list /etc/apt/sources.list.d/docker.sources
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | ${MAYBE_SUDO}tee /etc/apt/keyrings/docker.asc > /dev/null
${MAYBE_SUDO}chmod a+r /etc/apt/keyrings/docker.asc
CODENAME="$(. /etc/os-release && echo "${UBUNTU_CODENAME:-${VERSION_CODENAME:-}}")"
[[ -z "$CODENAME" ]] && { err "Could not detect Ubuntu version codename."; exit 1; }
${MAYBE_SUDO}tee /etc/apt/sources.list.d/docker.sources > /dev/null <<EOF
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: ${CODENAME}
Components: stable
Signed-By: /etc/apt/keyrings/docker.asc
EOF
info "Installing Docker Engine and Compose plugin..."
${MAYBE_SUDO}apt update
${MAYBE_SUDO}apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
if [[ -d /run/systemd/system ]]; then
info "Enabling and starting Docker..."
${MAYBE_SUDO}systemctl enable --now docker
else
err "No systemd. This script requires Ubuntu with systemd (e.g. DigitalOcean droplet)."
exit 1
fi
DOCKER_USER="${SUDO_USER:-${USER:-root}}"
if [[ "$DOCKER_USER" != "root" ]]; then
info "Adding $DOCKER_USER to docker group..."
${MAYBE_SUDO}usermod -aG docker "$DOCKER_USER"
fi
ok "Docker installed successfully."
echo ""
echo " Log out and back in (or run: newgrp docker) so the group change takes effect."
echo " Then verify with: docker compose version"
echo ""

File diff suppressed because it is too large Load Diff

View File

@@ -17,7 +17,7 @@ SERVER_ENV="$ROOT_DIR/server/.env"
WWW_ENV="$ROOT_DIR/www/.env.local"
MODEL="${LLM_MODEL:-qwen2.5:14b}"
OLLAMA_PORT="${OLLAMA_PORT:-11435}"
OLLAMA_PORT="${OLLAMA_PORT:-11434}"
OS="$(uname -s)"
@@ -35,98 +35,6 @@ err() { echo -e "${RED} ✗${NC} $*" >&2; }
# --- Helpers ---
dump_diagnostics() {
local failed_svc="${1:-}"
echo ""
err "========== DIAGNOSTICS =========="
err "Container status:"
compose_cmd ps -a --format "table {{.Name}}\t{{.Status}}" 2>/dev/null || true
echo ""
# Show logs for any container that exited
local stopped
stopped=$(compose_cmd ps -a --format '{{.Name}}\t{{.Status}}' 2>/dev/null \
| grep -iv 'up\|running' | awk -F'\t' '{print $1}' || true)
for c in $stopped; do
err "--- Logs for $c (exited/unhealthy) ---"
docker logs --tail 30 "$c" 2>&1 || true
echo ""
done
# If a specific service failed, always show its logs
if [[ -n "$failed_svc" ]]; then
err "--- Logs for $failed_svc (last 40) ---"
compose_cmd logs "$failed_svc" --tail 40 2>&1 || true
echo ""
# Try health check from inside the container as extra signal
err "--- Internal health check ($failed_svc) ---"
compose_cmd exec -T "$failed_svc" \
curl -sf http://localhost:1250/health 2>&1 || echo "(not reachable internally either)"
fi
err "================================="
}
trap 'dump_diagnostics' ERR
# Get the image ID for a compose service (works even when containers are not running).
svc_image_id() {
local svc="$1"
# Extract image name from compose config YAML, fall back to <project>-<service>
local img_name
img_name=$(compose_cmd config 2>/dev/null \
| sed -n "/^ ${svc}:/,/^ [a-z]/p" | grep '^\s*image:' | awk '{print $2}')
img_name="${img_name:-reflector-$svc}"
docker images -q "$img_name" 2>/dev/null | head -1
}
# Ensure images with build contexts are up-to-date.
# Docker layer cache makes this fast (~seconds) when source hasn't changed.
rebuild_images() {
local svc
for svc in web cpu; do
local old_id
old_id=$(svc_image_id "$svc")
old_id="${old_id:-<none>}"
info "Building $svc..."
compose_cmd build "$svc"
local new_id
new_id=$(svc_image_id "$svc")
if [[ "$old_id" == "$new_id" ]]; then
ok "$svc unchanged (${new_id:0:12})"
else
ok "$svc rebuilt (${old_id:0:12} -> ${new_id:0:12})"
fi
done
}
detect_lan_ip() {
# Returns the host's LAN IP — used for WebRTC ICE candidate rewriting.
case "$OS" in
Darwin)
# Try common interfaces: en0 (Wi-Fi), en1 (Ethernet)
for iface in en0 en1 en2 en3; do
local ip
ip=$(ipconfig getifaddr "$iface" 2>/dev/null || true)
if [[ -n "$ip" ]]; then
echo "$ip"
return
fi
done
;;
Linux)
ip route get 1.1.1.1 2>/dev/null | sed -n 's/.*src \([^ ]*\).*/\1/p'
return
;;
esac
# Fallback — empty means "not detected"
echo ""
}
wait_for_url() {
local url="$1" label="$2" retries="${3:-30}" interval="${4:-2}"
for i in $(seq 1 "$retries"); do
@@ -171,7 +79,7 @@ resolve_symlink() {
}
compose_cmd() {
local compose_files="-f $ROOT_DIR/docker-compose.standalone.yml"
local compose_files="-f $ROOT_DIR/docker-compose.yml -f $ROOT_DIR/docker-compose.standalone.yml"
if [[ "$OS" == "Linux" ]] && [[ -n "${OLLAMA_PROFILE:-}" ]]; then
docker compose $compose_files --profile "$OLLAMA_PROFILE" "$@"
else
@@ -205,7 +113,7 @@ step_llm() {
echo ""
# Pull model if not already present
if ollama list 2>/dev/null | awk '{print $1}' | grep -qxF "$MODEL"; then
if ollama list 2>/dev/null | awk '{print $1}' | grep -qx "$MODEL"; then
ok "Model $MODEL already pulled"
else
info "Pulling model $MODEL (this may take a while)..."
@@ -235,7 +143,7 @@ step_llm() {
echo ""
# Pull model inside container
if compose_cmd exec "$OLLAMA_SVC" ollama list 2>/dev/null | awk '{print $1}' | grep -qxF "$MODEL"; then
if compose_cmd exec "$OLLAMA_SVC" ollama list 2>/dev/null | awk '{print $1}' | grep -qx "$MODEL"; then
ok "Model $MODEL already pulled"
else
info "Pulling model $MODEL inside container (this may take a while)..."
@@ -285,17 +193,6 @@ ENVEOF
env_set "$SERVER_ENV" "LLM_MODEL" "$MODEL"
env_set "$SERVER_ENV" "LLM_API_KEY" "not-needed"
# WebRTC: detect LAN IP for ICE candidate rewriting (bridge networking)
local lan_ip
lan_ip=$(detect_lan_ip)
if [[ -n "$lan_ip" ]]; then
env_set "$SERVER_ENV" "WEBRTC_HOST" "$lan_ip"
ok "WebRTC host IP: $lan_ip"
else
warn "Could not detect LAN IP — WebRTC recording from other devices may not work"
warn "Set WEBRTC_HOST=<your-lan-ip> in server/.env manually"
fi
ok "Standalone vars set (LLM_URL=$LLM_URL_VALUE)"
}
@@ -308,24 +205,15 @@ step_storage() {
# Generate garage.toml from template (fill in RPC secret)
GARAGE_TOML="$ROOT_DIR/scripts/garage.toml"
GARAGE_TOML_RUNTIME="$ROOT_DIR/data/garage.toml"
mkdir -p "$ROOT_DIR/data"
if [[ -d "$GARAGE_TOML_RUNTIME" ]]; then
rm -rf "$GARAGE_TOML_RUNTIME"
fi
if [[ ! -f "$GARAGE_TOML_RUNTIME" ]]; then
mkdir -p "$ROOT_DIR/data"
RPC_SECRET=$(openssl rand -hex 32)
sed "s|__GARAGE_RPC_SECRET__|${RPC_SECRET}|" "$GARAGE_TOML" > "$GARAGE_TOML_RUNTIME"
fi
compose_cmd up -d garage
# Use /metrics for readiness — /health returns 503 until layout is applied
if ! wait_for_url "http://localhost:3903/metrics" "Garage admin API"; then
echo ""
err "Garage container logs:"
compose_cmd logs garage --tail 30 2>&1 || true
exit 1
fi
wait_for_url "http://localhost:3903/health" "Garage admin API"
echo ""
# Layout: get node ID, assign, apply (skip if already applied)
@@ -385,17 +273,11 @@ ENVEOF
ok "Created www/.env.local"
fi
# Caddyfile.standalone.example serves API at /v1, /health — use base URL
if [[ -n "${PRIMARY_IP:-}" ]]; then
BASE_URL="https://$PRIMARY_IP:3043"
else
BASE_URL="https://localhost:3043"
fi
env_set "$WWW_ENV" "SITE_URL" "$BASE_URL"
env_set "$WWW_ENV" "NEXTAUTH_URL" "$BASE_URL"
env_set "$WWW_ENV" "SITE_URL" "http://localhost:3000"
env_set "$WWW_ENV" "NEXTAUTH_URL" "http://localhost:3000"
env_set "$WWW_ENV" "NEXTAUTH_SECRET" "standalone-dev-secret-not-for-production"
env_set "$WWW_ENV" "API_URL" "$BASE_URL"
env_set "$WWW_ENV" "WEBSOCKET_URL" "auto"
env_set "$WWW_ENV" "API_URL" "http://localhost:1250"
env_set "$WWW_ENV" "WEBSOCKET_URL" "ws://localhost:1250"
env_set "$WWW_ENV" "SERVER_API_URL" "http://server:1250"
env_set "$WWW_ENV" "FEATURE_REQUIRE_LOGIN" "false"
@@ -411,7 +293,7 @@ step_services() {
# Check for port conflicts — stale processes silently shadow Docker port mappings.
# OrbStack/Docker Desktop bind ports for forwarding; ignore those PIDs.
local ports_ok=true
for port in 3043 3000 1250 5432 6379 3900 3903; do
for port in 3000 1250 5432 6379 3900 3903; do
local pids
pids=$(lsof -ti :"$port" 2>/dev/null || true)
for pid in $pids; do
@@ -431,24 +313,9 @@ step_services() {
warn "Continuing anyway (services will start but may be shadowed)"
fi
# Rebuild images if source has changed (Docker layer cache makes this fast when unchanged)
rebuild_images
# server runs alembic migrations on startup automatically (see runserver.sh)
compose_cmd up -d postgres redis garage cpu server worker beat web caddy
compose_cmd up -d postgres redis garage cpu server worker beat web
ok "Containers started"
# Quick sanity check — catch containers that exit immediately (bad image, missing file, etc.)
sleep 3
local exited
exited=$(compose_cmd ps -a --format '{{.Name}} {{.Status}}' 2>/dev/null \
| grep -i 'exit' || true)
if [[ -n "$exited" ]]; then
warn "Some containers exited immediately:"
echo "$exited" | while read -r line; do warn " $line"; done
dump_diagnostics
fi
info "Server is running migrations (alembic upgrade head)..."
}
@@ -478,49 +345,14 @@ step_health() {
warn "Check with: docker compose logs cpu"
fi
# Server may take a long time on first run — alembic migrations run before uvicorn starts.
# Use docker exec so this works regardless of network_mode or port mapping.
info "Waiting for Server API (first run includes database migrations)..."
local server_ok=false
for i in $(seq 1 90); do
# Check if container is still running
local svc_status
svc_status=$(compose_cmd ps server --format '{{.Status}}' 2>/dev/null || true)
if [[ -z "$svc_status" ]] || echo "$svc_status" | grep -qi 'exit'; then
echo ""
err "Server container exited unexpectedly"
dump_diagnostics server
exit 1
fi
# Health check from inside container (avoids host networking issues)
if compose_cmd exec -T server curl -sf http://localhost:1250/health > /dev/null 2>&1; then
server_ok=true
break
fi
echo -ne "\r Waiting for Server API... ($i/90)"
sleep 5
done
wait_for_url "http://localhost:1250/health" "Server API" 60 3
echo ""
if [[ "$server_ok" == "true" ]]; then
ok "Server API healthy"
else
err "Server API not ready after ~7 minutes"
dump_diagnostics server
exit 1
fi
ok "Server API healthy"
wait_for_url "http://localhost:3000" "Frontend" 90 3
echo ""
ok "Frontend responding"
# Caddy reverse proxy (self-signed TLS — curl needs -k)
if curl -sfk "https://localhost:3043" > /dev/null 2>&1; then
ok "Caddy proxy healthy (https://localhost:3043)"
else
warn "Caddy proxy not responding on https://localhost:3043"
warn "Check with: docker compose logs caddy"
fi
# Check LLM reachability from inside a container
if compose_cmd exec -T server \
curl -sf "$LLM_URL_VALUE/models" > /dev/null 2>&1; then
@@ -548,39 +380,6 @@ main() {
exit 1
fi
# Docker: Compose plugin, buildx, and daemon. On Ubuntu, auto-install if missing.
docker_ready() {
docker compose version 2>/dev/null | grep -qi compose \
&& docker buildx version &>/dev/null \
&& docker info &>/dev/null
}
if ! docker_ready; then
RAN_INSTALL=false
if [[ "$OS" == "Linux" ]] && [[ -f /etc/os-release ]] && (source /etc/os-release 2>/dev/null; [[ "${ID:-}" == "ubuntu" || "${ID_LIKE:-}" == *"ubuntu"* ]]); then
info "Docker not ready. Running install-docker-ubuntu.sh..."
"$SCRIPT_DIR/install-docker-ubuntu.sh" || true
RAN_INSTALL=true
[[ -d /run/systemd/system ]] && command -v systemctl &>/dev/null && systemctl start docker 2>/dev/null || true
sleep 2
fi
if ! docker_ready; then
# Docker may be installed but current shell lacks docker group (needs newgrp)
if [[ "$RAN_INSTALL" == "true" ]] && [[ $(id -u) -ne 0 ]] && command -v sg &>/dev/null && getent group docker &>/dev/null; then
info "Re-running with docker group..."
exec sg docker -c "$(printf '%q' "$0" && printf ' %q' "$@")"
fi
if [[ "$OS" == "Darwin" ]]; then
err "Docker not ready. Install Docker Desktop or OrbStack."
elif [[ "$OS" == "Linux" ]]; then
err "Docker not ready. Run: ./scripts/install-docker-ubuntu.sh"
err "Then run: newgrp docker (or log out and back in), then run this script again."
else
err "Docker not ready. Install Docker with Compose V2 and buildx."
fi
exit 1
fi
fi
# LLM_URL_VALUE is set by step_llm, used by later steps
LLM_URL_VALUE=""
@@ -590,57 +389,6 @@ main() {
# touch them so compose_cmd works before the steps that populate them.
touch "$SERVER_ENV" "$WWW_ENV"
# Ensure garage.toml exists before any compose up (step_llm starts all services including garage)
GARAGE_TOML="$ROOT_DIR/scripts/garage.toml"
GARAGE_TOML_RUNTIME="$ROOT_DIR/data/garage.toml"
mkdir -p "$ROOT_DIR/data"
if [[ -d "$GARAGE_TOML_RUNTIME" ]]; then
rm -rf "$GARAGE_TOML_RUNTIME"
fi
if [[ ! -f "$GARAGE_TOML_RUNTIME" ]]; then
RPC_SECRET=$(openssl rand -hex 32)
sed "s|__GARAGE_RPC_SECRET__|${RPC_SECRET}|" "$GARAGE_TOML" > "$GARAGE_TOML_RUNTIME"
fi
# Remove containers that may have bad mounts (was directory); force recreate
compose_cmd rm -f -s garage caddy 2>/dev/null || true
# Detect primary IP for droplet (used for Caddyfile, step_www_env, success message)
PRIMARY_IP=""
if [[ "$OS" == "Linux" ]]; then
PRIMARY_IP=$(hostname -I 2>/dev/null | awk '{print $1}' || true)
if [[ "$PRIMARY_IP" == "127."* ]] || [[ -z "$PRIMARY_IP" ]]; then
PRIMARY_IP=$(ip -4 route get 1 2>/dev/null | sed -n 's/.*src \([0-9.]*\).*/\1/p' || true)
fi
fi
# Ensure Caddyfile exists before any compose up (step_llm starts caddy)
# On droplet: explicit IP + localhost so Caddy provisions cert at startup (avoids on_demand/SNI issues)
CADDYFILE="$ROOT_DIR/Caddyfile"
if [[ -d "$CADDYFILE" ]]; then
rm -rf "$CADDYFILE"
fi
if [[ -n "$PRIMARY_IP" ]]; then
cat > "$CADDYFILE" << CADDYEOF
# Generated by setup-standalone.sh — explicit IP for droplet (provisions cert at startup)
https://$PRIMARY_IP, localhost {
tls internal
handle /v1/* {
reverse_proxy server:1250
}
handle /health {
reverse_proxy server:1250
}
handle {
reverse_proxy web:3000
}
}
CADDYEOF
ok "Created Caddyfile for $PRIMARY_IP and localhost"
elif [[ ! -f "$CADDYFILE" ]]; then
cp "$ROOT_DIR/Caddyfile.standalone.example" "$CADDYFILE"
fi
step_llm
echo ""
step_server_env
@@ -658,14 +406,8 @@ CADDYEOF
echo -e " ${GREEN}Reflector is running!${NC}"
echo "=========================================="
echo ""
if [[ -n "$PRIMARY_IP" ]]; then
echo " App: https://$PRIMARY_IP:3043 (accept self-signed cert in browser)"
echo " API: https://$PRIMARY_IP:3043/v1/"
echo " Local: https://localhost:3043"
else
echo " App: https://localhost:3043 (accept self-signed cert in browser)"
echo " API: https://localhost:3043/v1/"
fi
echo " Frontend: http://localhost:3000"
echo " API: http://localhost:1250"
echo ""
echo " To stop: docker compose down"
echo " To re-run: ./scripts/setup-standalone.sh"

View File

@@ -73,10 +73,10 @@ TRANSLATE_URL=https://monadical-sas--reflector-translator-web.modal.run
## Setup: ./scripts/setup-standalone.sh
## Mac: Ollama runs natively (Metal GPU). Containers reach it via host.docker.internal.
## Linux: docker compose --profile ollama-gpu up -d (or ollama-cpu for no GPU)
LLM_URL=http://host.docker.internal:11435/v1
LLM_URL=http://host.docker.internal:11434/v1
LLM_MODEL=qwen2.5:14b
LLM_API_KEY=not-needed
## Linux with containerized Ollama: LLM_URL=http://ollama:11435/v1
## Linux with containerized Ollama: LLM_URL=http://ollama:11434/v1
## --- Option B: Remote/cloud LLM ---
#LLM_API_KEY=sk-your-openai-api-key

View File

@@ -1,115 +0,0 @@
# =======================================================
# Reflector Self-Hosted Production — Backend Configuration
# Generated by: ./scripts/setup-selfhosted.sh
# Reference: server/reflector/settings.py
# =======================================================
# =======================================================
# Database & Infrastructure
# Pre-filled for Docker internal networking (docker-compose.selfhosted.yml)
# =======================================================
DATABASE_URL=postgresql+asyncpg://reflector:reflector@postgres:5432/reflector
REDIS_HOST=redis
REDIS_PORT=6379
CELERY_BROKER_URL=redis://redis:6379/1
CELERY_RESULT_BACKEND=redis://redis:6379/1
# Secret key — auto-generated by setup script
# Generate manually with: openssl rand -hex 32
SECRET_KEY=changeme-generate-a-secure-random-string
# =======================================================
# Authentication
# Disabled by default. Enable Authentik for multi-user access.
# See docsv2/selfhosted-production.md for setup instructions.
# =======================================================
AUTH_BACKEND=none
# AUTH_BACKEND=jwt
# AUTH_JWT_AUDIENCE=
# AUTH_BACKEND=password
# ADMIN_EMAIL=admin@localhost
# ADMIN_PASSWORD_HASH=pbkdf2:sha256:100000$<salt>$<hash>
# =======================================================
# Specialized Models (Transcription, Diarization, Translation)
# These run in the gpu/cpu container — NOT an LLM.
# The "modal" backend means "HTTP API client" — it talks to
# the self-hosted container, not Modal.com cloud.
# =======================================================
TRANSCRIPT_BACKEND=modal
TRANSCRIPT_URL=http://transcription:8000
TRANSCRIPT_MODAL_API_KEY=selfhosted
DIARIZATION_ENABLED=true
DIARIZATION_BACKEND=modal
DIARIZATION_URL=http://transcription:8000
TRANSLATION_BACKEND=modal
TRANSLATE_URL=http://transcription:8000
# HuggingFace token — optional, for gated models (e.g. pyannote).
# Falls back to public S3 model bundle if not set.
# HF_TOKEN=hf_xxxxx
# =======================================================
# LLM for Summarization & Topic Detection
# Only summaries and topics use an LLM. Everything else
# (transcription, diarization, translation) uses specialized models above.
#
# Supports any OpenAI-compatible endpoint.
# Auto-configured by setup script if using --ollama-gpu or --ollama-cpu.
# For --gpu or --cpu modes, you MUST configure an external LLM.
# =======================================================
# --- Option A: External OpenAI-compatible API ---
# LLM_URL=https://api.openai.com/v1
# LLM_API_KEY=sk-your-api-key
# LLM_MODEL=gpt-4o-mini
# --- Option B: Local Ollama (auto-set by --ollama-gpu/--ollama-cpu) ---
# LLM_URL=http://ollama:11435/v1
# LLM_API_KEY=not-needed
# LLM_MODEL=llama3.1
LLM_CONTEXT_WINDOW=16000
# =======================================================
# S3 Storage (REQUIRED)
# Where to store audio files and transcripts.
#
# Option A: Use --garage flag (auto-configured by setup script)
# Option B: Any S3-compatible endpoint (AWS, MinIO, etc.)
# Set TRANSCRIPT_STORAGE_AWS_ENDPOINT_URL for non-AWS endpoints.
# =======================================================
TRANSCRIPT_STORAGE_BACKEND=aws
TRANSCRIPT_STORAGE_AWS_ACCESS_KEY_ID=
TRANSCRIPT_STORAGE_AWS_SECRET_ACCESS_KEY=
TRANSCRIPT_STORAGE_AWS_BUCKET_NAME=reflector-media
TRANSCRIPT_STORAGE_AWS_REGION=us-east-1
# For non-AWS S3-compatible endpoints (Garage, MinIO, etc.):
# TRANSCRIPT_STORAGE_AWS_ENDPOINT_URL=http://garage:3900
# =======================================================
# Daily.co Live Rooms (Optional)
# Enable real-time meeting rooms with Daily.co integration.
# Requires a Daily.co account: https://www.daily.co/
# =======================================================
# DEFAULT_VIDEO_PLATFORM=daily
# DAILY_API_KEY=your-daily-api-key
# DAILY_SUBDOMAIN=your-subdomain
# DAILY_WEBHOOK_SECRET=your-daily-webhook-secret
# DAILYCO_STORAGE_AWS_BUCKET_NAME=reflector-dailyco
# DAILYCO_STORAGE_AWS_REGION=us-east-1
# DAILYCO_STORAGE_AWS_ROLE_ARN=arn:aws:iam::role/DailyCoAccess
# =======================================================
# Feature Flags
# =======================================================
PUBLIC_MODE=true
# FEATURE_ROOMS=true
# =======================================================
# Sentry (Optional)
# =======================================================
# SENTRY_DSN=

View File

@@ -1,74 +0,0 @@
"""add_change_seq_to_transcript
Revision ID: 623af934249a
Revises: 3aa20b96d963
Create Date: 2026-02-19 18:53:12.315440
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "623af934249a"
down_revision: Union[str, None] = "3aa20b96d963"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Sequence
op.execute("CREATE SEQUENCE IF NOT EXISTS transcript_change_seq;")
# Column (nullable first for backfill)
op.add_column("transcript", sa.Column("change_seq", sa.BigInteger(), nullable=True))
# Backfill existing rows with sequential values (ordered by created_at for determinism)
op.execute("""
UPDATE transcript SET change_seq = sub.seq FROM (
SELECT id, nextval('transcript_change_seq') AS seq
FROM transcript ORDER BY created_at ASC
) sub WHERE transcript.id = sub.id;
""")
# Now make NOT NULL
op.alter_column("transcript", "change_seq", nullable=False)
# Default for any inserts between now and trigger creation
op.alter_column(
"transcript",
"change_seq",
server_default=sa.text("nextval('transcript_change_seq')"),
)
# Trigger function
op.execute("""
CREATE OR REPLACE FUNCTION set_transcript_change_seq()
RETURNS TRIGGER AS $$
BEGIN
NEW.change_seq := nextval('transcript_change_seq');
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
""")
# Trigger (fires on every INSERT or UPDATE)
op.execute("""
CREATE TRIGGER trigger_transcript_change_seq
BEFORE INSERT OR UPDATE ON transcript
FOR EACH ROW
EXECUTE FUNCTION set_transcript_change_seq();
""")
# Index for efficient polling
op.create_index("idx_transcript_change_seq", "transcript", ["change_seq"])
def downgrade() -> None:
op.execute("DROP TRIGGER IF EXISTS trigger_transcript_change_seq ON transcript;")
op.execute("DROP FUNCTION IF EXISTS set_transcript_change_seq();")
op.drop_index("idx_transcript_change_seq", table_name="transcript")
op.drop_column("transcript", "change_seq")
op.execute("DROP SEQUENCE IF EXISTS transcript_change_seq;")

View File

@@ -1,25 +0,0 @@
"""add password_hash to user table
Revision ID: e1f093f7f124
Revises: 623af934249a
Create Date: 2026-02-19 00:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "e1f093f7f124"
down_revision: Union[str, None] = "623af934249a"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column("user", sa.Column("password_hash", sa.String(), nullable=True))
def downgrade() -> None:
op.drop_column("user", "password_hash")

View File

@@ -8,7 +8,6 @@ from prometheus_fastapi_instrumentator import Instrumentator
import reflector.auth # noqa
import reflector.db # noqa
from reflector.auth import router as auth_router
from reflector.events import subscribers_shutdown, subscribers_startup
from reflector.logger import logger
from reflector.metrics import metrics_init
@@ -38,13 +37,6 @@ try:
except ImportError:
sentry_sdk = None
# Patch aioice port range if configured (must happen before any RTCPeerConnection)
if settings.WEBRTC_PORT_RANGE:
from reflector.webrtc_ports import parse_port_range, patch_aioice_port_range
_min, _max = parse_port_range(settings.WEBRTC_PORT_RANGE)
patch_aioice_port_range(_min, _max)
# lifespan events
@asynccontextmanager
@@ -67,7 +59,7 @@ else:
logger.info("Sentry disabled")
# build app
app = FastAPI(lifespan=lifespan, root_path=settings.ROOT_PATH)
app = FastAPI(lifespan=lifespan)
app.add_middleware(
CORSMiddleware,
allow_credentials=settings.CORS_ALLOW_CREDENTIALS or False,
@@ -106,8 +98,6 @@ app.include_router(user_ws_router, prefix="/v1")
app.include_router(zulip_router, prefix="/v1")
app.include_router(whereby_router, prefix="/v1")
app.include_router(daily_router, prefix="/v1/daily")
if auth_router:
app.include_router(auth_router, prefix="/v1")
add_pagination(app)
# prepare celery

View File

@@ -4,9 +4,8 @@ from uuid import uuid4
from celery import current_task
from reflector.db import _database_context, get_database
from reflector.db import get_database
from reflector.llm import llm_session_id
from reflector.ws_manager import reset_ws_manager
def asynctask(f):
@@ -21,14 +20,6 @@ def asynctask(f):
return await f(*args, **kwargs)
finally:
await database.disconnect()
_database_context.set(None)
if current_task:
# Reset cached connections before each Celery task.
# Each asyncio.run() creates a new event loop, making connections
# from previous tasks stale ("Future attached to a different loop").
_database_context.set(None)
reset_ws_manager()
coro = run_with_db()
if current_task:

View File

@@ -12,8 +12,3 @@ AccessTokenInfo = auth_module.AccessTokenInfo
authenticated = auth_module.authenticated
current_user = auth_module.current_user
current_user_optional = auth_module.current_user_optional
parse_ws_bearer_token = auth_module.parse_ws_bearer_token
current_user_ws_optional = auth_module.current_user_ws_optional
# Optional router (e.g. for /auth/login in password backend)
router = getattr(auth_module, "router", None)

View File

@@ -1,9 +1,6 @@
from typing import TYPE_CHECKING, Annotated, List, Optional
from typing import Annotated, List, Optional
from fastapi import Depends, HTTPException
if TYPE_CHECKING:
from fastapi import WebSocket
from fastapi.security import APIKeyHeader, OAuth2PasswordBearer
from jose import JWTError, jwt
from pydantic import BaseModel
@@ -127,20 +124,3 @@ async def current_user_optional(
jwtauth: JWTAuth = Depends(),
):
return await _authenticate_user(jwt_token, api_key, jwtauth)
def parse_ws_bearer_token(
websocket: "WebSocket",
) -> tuple[Optional[str], Optional[str]]:
raw = websocket.headers.get("sec-websocket-protocol") or ""
parts = [p.strip() for p in raw.split(",") if p.strip()]
if len(parts) >= 2 and parts[0].lower() == "bearer":
return parts[1], "bearer"
return None, None
async def current_user_ws_optional(websocket: "WebSocket") -> Optional[UserInfo]:
token, _ = parse_ws_bearer_token(websocket)
if not token:
return None
return await _authenticate_user(token, None, JWTAuth())

View File

@@ -19,11 +19,3 @@ def current_user():
def current_user_optional():
return None
def parse_ws_bearer_token(websocket):
return None, None
async def current_user_ws_optional(websocket):
return None

View File

@@ -1,198 +0,0 @@
"""Password-based authentication backend for selfhosted deployments.
Issues HS256 JWTs signed with settings.SECRET_KEY. Provides a POST /auth/login
endpoint for email/password authentication.
"""
import time
from collections import defaultdict
from datetime import datetime, timedelta, timezone
from typing import TYPE_CHECKING, Annotated, Optional
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.security import APIKeyHeader, OAuth2PasswordBearer
from jose import JWTError, jwt
from pydantic import BaseModel
from reflector.auth.password_utils import verify_password
from reflector.db.user_api_keys import user_api_keys_controller
from reflector.db.users import user_controller
from reflector.logger import logger
from reflector.settings import settings
if TYPE_CHECKING:
from fastapi import WebSocket
# --- FastAPI security schemes (same pattern as auth_jwt.py) ---
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/v1/auth/login", auto_error=False)
api_key_header = APIKeyHeader(name="X-API-Key", auto_error=False)
# --- JWT configuration ---
JWT_ALGORITHM = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES = 60 * 24 # 24 hours
# --- Rate limiting (in-memory) ---
_login_attempts: dict[str, list[float]] = defaultdict(list)
RATE_LIMIT_WINDOW = 300 # 5 minutes
RATE_LIMIT_MAX = 10 # max attempts per window
def _check_rate_limit(key: str) -> bool:
"""Return True if request is allowed, False if rate-limited."""
now = time.monotonic()
attempts = _login_attempts[key]
_login_attempts[key] = [t for t in attempts if now - t < RATE_LIMIT_WINDOW]
if len(_login_attempts[key]) >= RATE_LIMIT_MAX:
return False
_login_attempts[key].append(now)
return True
# --- Pydantic models ---
class UserInfo(BaseModel):
sub: str
email: Optional[str] = None
def __getitem__(self, key):
return getattr(self, key)
class AccessTokenInfo(BaseModel):
exp: Optional[int] = None
sub: Optional[str] = None
class LoginRequest(BaseModel):
email: str
password: str
class LoginResponse(BaseModel):
access_token: str
token_type: str = "bearer"
expires_in: int
# --- JWT token creation and verification ---
def _create_access_token(user_id: str, email: str) -> tuple[str, int]:
"""Create an HS256 JWT. Returns (token, expires_in_seconds)."""
expires_delta = timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
expire = datetime.now(timezone.utc) + expires_delta
payload = {
"sub": user_id,
"email": email,
"exp": expire,
}
token = jwt.encode(payload, settings.SECRET_KEY, algorithm=JWT_ALGORITHM)
return token, int(expires_delta.total_seconds())
def _verify_token(token: str) -> dict:
"""Verify and decode an HS256 JWT."""
return jwt.decode(token, settings.SECRET_KEY, algorithms=[JWT_ALGORITHM])
# --- Authentication logic (mirrors auth_jwt._authenticate_user) ---
async def _authenticate_user(
jwt_token: Optional[str],
api_key: Optional[str],
) -> UserInfo | None:
user_infos: list[UserInfo] = []
if api_key:
user_api_key = await user_api_keys_controller.verify_key(api_key)
if user_api_key:
user_infos.append(UserInfo(sub=user_api_key.user_id, email=None))
if jwt_token:
try:
payload = _verify_token(jwt_token)
user_id = payload["sub"]
email = payload.get("email")
user_infos.append(UserInfo(sub=user_id, email=email))
except JWTError as e:
logger.error(f"JWT error: {e}")
raise HTTPException(status_code=401, detail="Invalid authentication")
if len(user_infos) == 0:
return None
if len(set(x.sub for x in user_infos)) > 1:
raise HTTPException(
status_code=401,
detail="Invalid authentication: more than one user provided",
)
return user_infos[0]
# --- FastAPI dependencies (exported, required by auth/__init__.py) ---
def authenticated(token: Annotated[str, Depends(oauth2_scheme)]):
if token is None:
raise HTTPException(status_code=401, detail="Not authenticated")
return None
async def current_user(
jwt_token: Annotated[Optional[str], Depends(oauth2_scheme)],
api_key: Annotated[Optional[str], Depends(api_key_header)],
):
user = await _authenticate_user(jwt_token, api_key)
if user is None:
raise HTTPException(status_code=401, detail="Not authenticated")
return user
async def current_user_optional(
jwt_token: Annotated[Optional[str], Depends(oauth2_scheme)],
api_key: Annotated[Optional[str], Depends(api_key_header)],
):
return await _authenticate_user(jwt_token, api_key)
# --- WebSocket auth (same pattern as auth_jwt.py) ---
def parse_ws_bearer_token(
websocket: "WebSocket",
) -> tuple[Optional[str], Optional[str]]:
raw = websocket.headers.get("sec-websocket-protocol") or ""
parts = [p.strip() for p in raw.split(",") if p.strip()]
if len(parts) >= 2 and parts[0].lower() == "bearer":
return parts[1], "bearer"
return None, None
async def current_user_ws_optional(websocket: "WebSocket") -> Optional[UserInfo]:
token, _ = parse_ws_bearer_token(websocket)
if not token:
return None
return await _authenticate_user(token, None)
# --- Login router ---
router = APIRouter(prefix="/auth", tags=["auth"])
@router.post("/login", response_model=LoginResponse)
async def login(request: Request, body: LoginRequest):
client_ip = request.client.host if request.client else "unknown"
if not _check_rate_limit(client_ip):
raise HTTPException(
status_code=429,
detail="Too many login attempts. Try again later.",
)
user = await user_controller.get_by_email(body.email)
if not user or not user.password_hash:
print("invalid email")
raise HTTPException(status_code=401, detail="Invalid email or password")
if not verify_password(body.password, user.password_hash):
print("invalid pass")
raise HTTPException(status_code=401, detail="Invalid email or password")
access_token, expires_in = _create_access_token(user.id, user.email)
return LoginResponse(
access_token=access_token,
token_type="bearer",
expires_in=expires_in,
)

View File

@@ -1,41 +0,0 @@
"""Password hashing utilities using PBKDF2-SHA256 (stdlib only)."""
import hashlib
import hmac
import os
PBKDF2_ITERATIONS = 100_000
SALT_LENGTH = 16 # bytes, hex-encoded to 32 chars
def hash_password(password: str) -> str:
"""Hash a password using PBKDF2-SHA256 with a random salt.
Format: pbkdf2:sha256:<iterations>$<salt_hex>$<hash_hex>
"""
salt = os.urandom(SALT_LENGTH).hex()
dk = hashlib.pbkdf2_hmac(
"sha256",
password.encode("utf-8"),
salt.encode("utf-8"),
PBKDF2_ITERATIONS,
)
return f"pbkdf2:sha256:{PBKDF2_ITERATIONS}${salt}${dk.hex()}"
def verify_password(password: str, password_hash: str) -> bool:
"""Verify a password against its hash using constant-time comparison."""
try:
header, salt, stored_hash = password_hash.split("$", 2)
_, algo, iterations_str = header.split(":")
iterations = int(iterations_str)
dk = hashlib.pbkdf2_hmac(
algo,
password.encode("utf-8"),
salt.encode("utf-8"),
iterations,
)
return hmac.compare_digest(dk.hex(), stored_hash)
except (ValueError, AttributeError):
return False

View File

@@ -151,7 +151,6 @@ class SearchResultDB(BaseModel):
title: str | None = None
source_kind: SourceKind
room_id: str | None = None
change_seq: int | None = None
rank: float = Field(..., ge=0, le=1)
@@ -174,7 +173,6 @@ class SearchResult(BaseModel):
total_match_count: NonNegativeInt = Field(
default=0, description="Total number of matches found in the transcript"
)
change_seq: int | None = None
@field_serializer("created_at", when_used="json")
def serialize_datetime(self, dt: datetime) -> str:
@@ -358,7 +356,6 @@ class SearchController:
transcripts.c.user_id,
transcripts.c.room_id,
transcripts.c.source_kind,
transcripts.c.change_seq,
transcripts.c.webvtt,
transcripts.c.long_summary,
sqlalchemy.case(

View File

@@ -5,10 +5,7 @@ import shutil
from contextlib import asynccontextmanager
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import TYPE_CHECKING, Any, Literal, Sequence
if TYPE_CHECKING:
from reflector.ws_events import TranscriptEventName
from typing import Any, Literal, Sequence
import sqlalchemy
from fastapi import HTTPException
@@ -35,8 +32,6 @@ class SourceKind(enum.StrEnum):
FILE = enum.auto()
transcript_change_seq = sqlalchemy.Sequence("transcript_change_seq", metadata=metadata)
transcripts = sqlalchemy.Table(
"transcript",
metadata,
@@ -91,12 +86,6 @@ transcripts = sqlalchemy.Table(
sqlalchemy.Column("webvtt", sqlalchemy.Text),
# Hatchet workflow run ID for resumption of failed workflows
sqlalchemy.Column("workflow_run_id", sqlalchemy.String),
sqlalchemy.Column(
"change_seq",
sqlalchemy.BigInteger,
transcript_change_seq,
server_default=transcript_change_seq.next_value(),
),
sqlalchemy.Index("idx_transcript_recording_id", "recording_id"),
sqlalchemy.Index("idx_transcript_user_id", "user_id"),
sqlalchemy.Index("idx_transcript_created_at", "created_at"),
@@ -195,7 +184,7 @@ class TranscriptWaveform(BaseModel):
class TranscriptEvent(BaseModel):
event: str # Typed at call sites via ws_events.TranscriptEventName; str here for DB compat
event: str
data: dict
@@ -237,7 +226,6 @@ class Transcript(BaseModel):
audio_deleted: bool | None = None
webvtt: str | None = None
workflow_run_id: str | None = None # Hatchet workflow run ID for resumption
change_seq: int | None = None
@field_serializer("created_at", when_used="json")
def serialize_datetime(self, dt: datetime) -> str:
@@ -245,9 +233,7 @@ class Transcript(BaseModel):
dt = dt.replace(tzinfo=timezone.utc)
return dt.isoformat()
def add_event(
self, event: "TranscriptEventName", data: BaseModel
) -> TranscriptEvent:
def add_event(self, event: str, data: BaseModel) -> TranscriptEvent:
ev = TranscriptEvent(event=event, data=data.model_dump())
self.events.append(ev)
return ev
@@ -390,7 +376,6 @@ class TranscriptController:
source_kind: SourceKind | None = None,
room_id: str | None = None,
search_term: str | None = None,
change_seq_from: int | None = None,
return_query: bool = False,
exclude_columns: list[str] = [
"topics",
@@ -411,7 +396,6 @@ class TranscriptController:
- `filter_recording`: filter out transcripts that are currently recording
- `room_id`: filter transcripts by room ID
- `search_term`: filter transcripts by search term
- `change_seq_from`: filter transcripts with change_seq > this value
"""
query = transcripts.select().join(
@@ -434,9 +418,6 @@ class TranscriptController:
if search_term:
query = query.where(transcripts.c.title.ilike(f"%{search_term}%"))
if change_seq_from is not None:
query = query.where(transcripts.c.change_seq > change_seq_from)
# Exclude heavy JSON columns from list queries
transcript_columns = [
col for col in transcripts.c if col.name not in exclude_columns
@@ -450,10 +431,9 @@ class TranscriptController:
)
if order_by is not None:
field = getattr(transcripts.c, order_by[1:])
if order_by.startswith("-"):
field = getattr(transcripts.c, order_by[1:]).desc()
else:
field = getattr(transcripts.c, order_by)
field = field.desc()
query = query.order_by(field)
if filter_empty:
@@ -708,7 +688,7 @@ class TranscriptController:
async def append_event(
self,
transcript: Transcript,
event: "TranscriptEventName",
event: str,
data: Any,
) -> TranscriptEvent:
"""

View File

@@ -1,4 +1,4 @@
"""User table for storing user information."""
"""User table for storing Authentik user information."""
from datetime import datetime, timezone
@@ -15,7 +15,6 @@ users = sqlalchemy.Table(
sqlalchemy.Column("id", sqlalchemy.String, primary_key=True),
sqlalchemy.Column("email", sqlalchemy.String, nullable=False),
sqlalchemy.Column("authentik_uid", sqlalchemy.String, nullable=False),
sqlalchemy.Column("password_hash", sqlalchemy.String, nullable=True),
sqlalchemy.Column("created_at", sqlalchemy.DateTime(timezone=True), nullable=False),
sqlalchemy.Column("updated_at", sqlalchemy.DateTime(timezone=True), nullable=False),
sqlalchemy.Index("idx_user_authentik_uid", "authentik_uid", unique=True),
@@ -27,7 +26,6 @@ class User(BaseModel):
id: NonEmptyString = Field(default_factory=generate_uuid4)
email: NonEmptyString
authentik_uid: NonEmptyString
password_hash: str | None = None
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
updated_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
@@ -53,29 +51,22 @@ class UserController:
@staticmethod
async def create_or_update(
id: NonEmptyString,
authentik_uid: NonEmptyString,
email: NonEmptyString,
password_hash: str | None = None,
id: NonEmptyString, authentik_uid: NonEmptyString, email: NonEmptyString
) -> User:
existing = await UserController.get_by_authentik_uid(authentik_uid)
now = datetime.now(timezone.utc)
if existing:
update_values: dict = {"email": email, "updated_at": now}
if password_hash is not None:
update_values["password_hash"] = password_hash
query = (
users.update()
.where(users.c.authentik_uid == authentik_uid)
.values(**update_values)
.values(email=email, updated_at=now)
)
await get_database().execute(query)
return User(
id=existing.id,
authentik_uid=authentik_uid,
email=email,
password_hash=password_hash or existing.password_hash,
created_at=existing.created_at,
updated_at=now,
)
@@ -84,7 +75,6 @@ class UserController:
id=id,
authentik_uid=authentik_uid,
email=email,
password_hash=password_hash,
created_at=now,
updated_at=now,
)
@@ -92,16 +82,6 @@ class UserController:
await get_database().execute(query)
return user
@staticmethod
async def set_password_hash(user_id: NonEmptyString, password_hash: str) -> None:
now = datetime.now(timezone.utc)
query = (
users.update()
.where(users.c.id == user_id)
.values(password_hash=password_hash, updated_at=now)
)
await get_database().execute(query)
@staticmethod
async def list_all() -> list[User]:
query = users.select().order_by(users.c.created_at.desc())

View File

@@ -12,11 +12,10 @@ import structlog
from reflector.db.transcripts import Transcript, TranscriptEvent, transcripts_controller
from reflector.utils.string import NonEmptyString
from reflector.ws_events import TranscriptEventName
from reflector.ws_manager import get_ws_manager
# Events that should also be sent to user room (matches Celery behavior)
USER_ROOM_EVENTS: set[TranscriptEventName] = {"STATUS", "FINAL_TITLE", "DURATION"}
USER_ROOM_EVENTS = {"STATUS", "FINAL_TITLE", "DURATION"}
async def broadcast_event(
@@ -82,7 +81,8 @@ async def set_status_and_broadcast(
async def append_event_and_broadcast(
transcript_id: NonEmptyString,
transcript: Transcript,
event_name: TranscriptEventName,
event_name: NonEmptyString,
# TODO proper dictionary event => type
data: Any,
logger: structlog.BoundLogger,
) -> TranscriptEvent:

View File

@@ -71,7 +71,7 @@ async def detect_chunk_topic(input: TopicChunkInput, ctx: Context) -> TopicChunk
from reflector.settings import settings # noqa: PLC0415
from reflector.utils.text import clean_title # noqa: PLC0415
llm = LLM(settings=settings, temperature=0.9)
llm = LLM(settings=settings, temperature=0.9, max_tokens=500)
prompt = TOPIC_PROMPT.format(text=input.chunk_text)
response = await llm.get_structured_response(

View File

@@ -1,23 +1,42 @@
import logging
from contextvars import ContextVar
from typing import Type, TypeVar
from typing import Generic, Type, TypeVar
from uuid import uuid4
from llama_index.core import Settings
from llama_index.core.prompts import PromptTemplate
from llama_index.core.output_parsers import PydanticOutputParser
from llama_index.core.response_synthesizers import TreeSummarize
from llama_index.core.workflow import (
Context,
Event,
StartEvent,
StopEvent,
Workflow,
step,
)
from llama_index.llms.openai_like import OpenAILike
from pydantic import BaseModel, ValidationError
from workflows.errors import WorkflowTimeoutError
from reflector.utils.retry import retry
T = TypeVar("T", bound=BaseModel)
OutputT = TypeVar("OutputT", bound=BaseModel)
# Session ID for LiteLLM request grouping - set per processing run
llm_session_id: ContextVar[str | None] = ContextVar("llm_session_id", default=None)
logger = logging.getLogger(__name__)
STRUCTURED_RESPONSE_PROMPT_TEMPLATE = """
Based on the following analysis, provide the information in the requested JSON format:
Analysis:
{analysis}
{format_instructions}
"""
class LLMParseError(Exception):
"""Raised when LLM output cannot be parsed after retries."""
@@ -31,10 +50,159 @@ class LLMParseError(Exception):
)
class LLM:
class ExtractionDone(Event):
"""Event emitted when LLM JSON formatting completes."""
output: str
class ValidationErrorEvent(Event):
"""Event emitted when validation fails."""
error: str
wrong_output: str
class StructuredOutputWorkflow(Workflow, Generic[OutputT]):
"""Workflow for structured output extraction with validation retry.
This workflow handles parse/validation retries only. Network error retries
are handled internally by Settings.llm (OpenAILike max_retries=3).
The caller should NOT wrap this workflow in additional retry logic.
"""
def __init__(
self, settings, temperature: float = 0.4, max_tokens: int | None = None
self,
output_cls: Type[OutputT],
max_retries: int = 3,
**kwargs,
):
super().__init__(**kwargs)
self.output_cls: Type[OutputT] = output_cls
self.max_retries = max_retries
self.output_parser = PydanticOutputParser(output_cls)
@step
async def extract(
self, ctx: Context, ev: StartEvent | ValidationErrorEvent
) -> StopEvent | ExtractionDone:
"""Extract structured data from text using two-step LLM process.
Step 1 (first call only): TreeSummarize generates text analysis
Step 2 (every call): Settings.llm.acomplete formats analysis as JSON
"""
current_retries = await ctx.store.get("retries", default=0)
await ctx.store.set("retries", current_retries + 1)
if current_retries >= self.max_retries:
last_error = await ctx.store.get("last_error", default=None)
logger.error(
f"Max retries ({self.max_retries}) reached for {self.output_cls.__name__}"
)
return StopEvent(result={"error": last_error, "attempts": current_retries})
if isinstance(ev, StartEvent):
# First call: run TreeSummarize to get analysis, store in context
prompt = ev.get("prompt")
texts = ev.get("texts")
tone_name = ev.get("tone_name")
if not prompt or not isinstance(texts, list):
raise ValueError(
"StartEvent must contain 'prompt' (str) and 'texts' (list)"
)
summarizer = TreeSummarize(verbose=False)
analysis = await summarizer.aget_response(
prompt, texts, tone_name=tone_name
)
await ctx.store.set("analysis", str(analysis))
reflection = ""
else:
# Retry: reuse analysis from context
analysis = await ctx.store.get("analysis")
if not analysis:
raise RuntimeError("Internal error: analysis not found in context")
wrong_output = ev.wrong_output
if len(wrong_output) > 2000:
wrong_output = wrong_output[:2000] + "... [truncated]"
reflection = (
f"\n\nYour previous response could not be parsed:\n{wrong_output}\n\n"
f"Error:\n{ev.error}\n\n"
"Please try again. Return ONLY valid JSON matching the schema above, "
"with no markdown formatting or extra text."
)
# Step 2: Format analysis as JSON using LLM completion
format_instructions = self.output_parser.format(
"Please structure the above information in the following JSON format:"
)
json_prompt = STRUCTURED_RESPONSE_PROMPT_TEMPLATE.format(
analysis=analysis,
format_instructions=format_instructions + reflection,
)
# Network retries handled by OpenAILike (max_retries=3)
# response_format enables grammar-based constrained decoding on backends
# that support it (DMR/llama.cpp, vLLM, Ollama, OpenAI).
response = await Settings.llm.acomplete(
json_prompt,
response_format={
"type": "json_schema",
"json_schema": {
"name": self.output_cls.__name__,
"schema": self.output_cls.model_json_schema(),
},
},
)
return ExtractionDone(output=response.text)
@step
async def validate(
self, ctx: Context, ev: ExtractionDone
) -> StopEvent | ValidationErrorEvent:
"""Validate extracted output against Pydantic schema."""
raw_output = ev.output
retries = await ctx.store.get("retries", default=0)
try:
parsed = self.output_parser.parse(raw_output)
if retries > 1:
logger.info(
f"LLM parse succeeded on attempt {retries}/{self.max_retries} "
f"for {self.output_cls.__name__}"
)
return StopEvent(result={"success": parsed})
except (ValidationError, ValueError) as e:
error_msg = self._format_error(e, raw_output)
await ctx.store.set("last_error", error_msg)
logger.error(
f"LLM parse error (attempt {retries}/{self.max_retries}): "
f"{type(e).__name__}: {e}\nRaw response: {raw_output[:500]}"
)
return ValidationErrorEvent(
error=error_msg,
wrong_output=raw_output,
)
def _format_error(self, error: Exception, raw_output: str) -> str:
"""Format error for LLM feedback."""
if isinstance(error, ValidationError):
error_messages = []
for err in error.errors():
field = ".".join(str(loc) for loc in err["loc"])
error_messages.append(f"- {err['msg']} in field '{field}'")
return "Schema validation errors:\n" + "\n".join(error_messages)
else:
return f"Parse error: {str(error)}"
class LLM:
def __init__(self, settings, temperature: float = 0.4, max_tokens: int = 2048):
self.settings_obj = settings
self.model_name = settings.LLM_MODEL
self.url = settings.LLM_URL
@@ -55,10 +223,9 @@ class LLM:
api_key=self.api_key,
context_window=self.context_window,
is_chat_model=True,
is_function_calling_model=True,
is_function_calling_model=False,
temperature=self.temperature,
max_tokens=self.max_tokens,
timeout=self.settings_obj.LLM_REQUEST_TIMEOUT,
additional_kwargs={"extra_body": {"litellm_session_id": session_id}},
)
@@ -78,91 +245,36 @@ class LLM:
tone_name: str | None = None,
timeout: int | None = None,
) -> T:
"""Get structured output from LLM using tool-call with reflection retry.
"""Get structured output from LLM with validation retry via Workflow."""
if timeout is None:
timeout = self.settings_obj.LLM_STRUCTURED_RESPONSE_TIMEOUT
Uses astructured_predict (function-calling / tool-call mode) for the
first attempt. On ValidationError or parse failure the wrong output
and error are fed back as a reflection prompt and the call is retried
up to LLM_PARSE_MAX_RETRIES times.
The outer retry() wrapper handles transient network errors with
exponential back-off.
"""
max_retries = self.settings_obj.LLM_PARSE_MAX_RETRIES
async def _call_with_reflection():
# Build full prompt: instruction + source texts
if texts:
texts_block = "\n\n".join(texts)
full_prompt = f"{prompt}\n\n{texts_block}"
else:
full_prompt = prompt
prompt_tmpl = PromptTemplate("{user_prompt}")
last_error: str | None = None
for attempt in range(1, max_retries + 2): # +2: first try + retries
try:
if attempt == 1:
result = await Settings.llm.astructured_predict(
output_cls, prompt_tmpl, user_prompt=full_prompt
)
else:
reflection_tmpl = PromptTemplate(
"{user_prompt}\n\n{reflection}"
)
result = await Settings.llm.astructured_predict(
output_cls,
reflection_tmpl,
user_prompt=full_prompt,
reflection=reflection,
)
if attempt > 1:
logger.info(
f"LLM structured_predict succeeded on attempt "
f"{attempt}/{max_retries + 1} for {output_cls.__name__}"
)
return result
except (ValidationError, ValueError) as e:
wrong_output = str(e)
if len(wrong_output) > 2000:
wrong_output = wrong_output[:2000] + "... [truncated]"
last_error = self._format_validation_error(e)
reflection = (
f"Your previous response could not be parsed.\n\n"
f"Error:\n{last_error}\n\n"
"Please try again and return valid data matching the schema."
)
logger.error(
f"LLM parse error (attempt {attempt}/{max_retries + 1}): "
f"{type(e).__name__}: {e}\n"
f"Raw response: {wrong_output[:500]}"
)
raise LLMParseError(
async def run_workflow():
workflow = StructuredOutputWorkflow(
output_cls=output_cls,
error_msg=last_error or "Max retries exceeded",
attempts=max_retries + 1,
max_retries=self.settings_obj.LLM_PARSE_MAX_RETRIES + 1,
timeout=timeout,
)
return await retry(_call_with_reflection)(
result = await workflow.run(
prompt=prompt,
texts=texts,
tone_name=tone_name,
)
if "error" in result:
error_msg = result["error"] or "Max retries exceeded"
raise LLMParseError(
output_cls=output_cls,
error_msg=error_msg,
attempts=result.get("attempts", 0),
)
return result["success"]
return await retry(run_workflow)(
retry_attempts=3,
retry_backoff_interval=1.0,
retry_backoff_max=30.0,
retry_ignore_exc_types=(ConnectionError, TimeoutError, OSError),
retry_ignore_exc_types=(WorkflowTimeoutError,),
)
@staticmethod
def _format_validation_error(error: Exception) -> str:
"""Format a validation/parse error for LLM reflection feedback."""
if isinstance(error, ValidationError):
error_messages = []
for err in error.errors():
field = ".".join(str(loc) for loc in err["loc"])
error_messages.append(f"- {err['msg']} in field '{field}'")
return "Schema validation errors:\n" + "\n".join(error_messages)
return f"Parse error: {str(error)}"

View File

@@ -62,8 +62,6 @@ from reflector.processors.types import (
from reflector.processors.types import Transcript as TranscriptProcessorType
from reflector.settings import settings
from reflector.storage import get_transcripts_storage
from reflector.views.transcripts import GetTranscriptTopic
from reflector.ws_events import TranscriptEventName
from reflector.ws_manager import WebsocketManager, get_ws_manager
from reflector.zulip import (
get_zulip_message,
@@ -91,11 +89,7 @@ def broadcast_to_sockets(func):
if transcript and transcript.user_id:
# Emit only relevant events to the user room to avoid noisy updates.
# Allowed: STATUS, FINAL_TITLE, DURATION. All are prefixed with TRANSCRIPT_
allowed_user_events: set[TranscriptEventName] = {
"STATUS",
"FINAL_TITLE",
"DURATION",
}
allowed_user_events = {"STATUS", "FINAL_TITLE", "DURATION"}
if resp.event in allowed_user_events:
await self.ws_manager.send_json(
room_id=f"user:{transcript.user_id}",
@@ -250,14 +244,13 @@ class PipelineMainBase(PipelineRunner[PipelineMessage], Generic[PipelineMessage]
)
if isinstance(data, TitleSummaryWithIdProcessorType):
topic.id = data.id
get_topic = GetTranscriptTopic.from_transcript_topic(topic)
async with self.transaction():
transcript = await self.get_transcript()
await transcripts_controller.upsert_topic(transcript, topic)
return await transcripts_controller.append_event(
transcript=transcript,
event="TOPIC",
data=get_topic,
data=topic,
)
@broadcast_to_sockets

View File

@@ -39,7 +39,7 @@ class TranscriptFinalTitleProcessor(Processor):
def __init__(self, **kwargs):
super().__init__(**kwargs)
self.chunks: list[TitleSummary] = []
self.llm = LLM(settings=settings, temperature=0.5)
self.llm = LLM(settings=settings, temperature=0.5, max_tokens=200)
async def _push(self, data: TitleSummary):
self.chunks.append(data)

View File

@@ -14,12 +14,10 @@ class TopicResponse(BaseModel):
title: str = Field(
description="A descriptive title for the topic being discussed",
validation_alias=AliasChoices("title", "Title"),
min_length=8,
)
summary: str = Field(
description="A concise 1-2 sentence summary of the discussion",
validation_alias=AliasChoices("summary", "Summary"),
min_length=8,
)
@@ -37,7 +35,7 @@ class TranscriptTopicDetectorProcessor(Processor):
super().__init__(**kwargs)
self.transcript = None
self.min_transcript_length = min_transcript_length
self.llm = LLM(settings=settings, temperature=0.9)
self.llm = LLM(settings=settings, temperature=0.9, max_tokens=500)
async def _push(self, data: Transcript):
if self.transcript is None:

View File

@@ -12,17 +12,6 @@ class Settings(BaseSettings):
extra="ignore",
)
ROOT_PATH: str = "/"
# WebRTC port range for ICE candidates (e.g. "50000-50100").
# When set, monkey-patches aioice to bind UDP sockets within this range,
# allowing Docker port mapping instead of network_mode: host.
WEBRTC_PORT_RANGE: str | None = None
# Host IP or hostname to advertise in ICE candidates instead of the
# container's internal IP. Use "host.docker.internal" in Docker with
# extra_hosts, or a specific LAN IP. Resolved at connection time.
WEBRTC_HOST: str | None = None
# CORS
UI_BASE_URL: str = "http://localhost:3000"
CORS_ORIGIN: str = "*"
@@ -87,7 +76,6 @@ class Settings(BaseSettings):
LLM_URL: str | None = None
LLM_API_KEY: str | None = None
LLM_CONTEXT_WINDOW: int = 16000
LLM_REQUEST_TIMEOUT: float = 300.0 # HTTP request timeout for LLM calls (seconds)
LLM_PARSE_MAX_RETRIES: int = (
3 # Max retries for JSON/validation errors (total attempts = retries + 1)
@@ -113,7 +101,7 @@ class Settings(BaseSettings):
# Sentry
SENTRY_DSN: str | None = None
# User authentication (none, jwt, password)
# User authentication (none, jwt)
AUTH_BACKEND: str = "none"
# User authentication using JWT
@@ -121,10 +109,6 @@ class Settings(BaseSettings):
AUTH_JWT_PUBLIC_KEY: str | None = "authentik.monadical.com_public.pem"
AUTH_JWT_AUDIENCE: str | None = None
# User authentication using password (selfhosted)
ADMIN_EMAIL: str | None = None
ADMIN_PASSWORD_HASH: str | None = None
PUBLIC_MODE: bool = False
PUBLIC_DATA_RETENTION_DAYS: PositiveInt = 7
@@ -158,9 +142,6 @@ class Settings(BaseSettings):
WHEREBY_WEBHOOK_SECRET: str | None = None
AWS_PROCESS_RECORDING_QUEUE_URL: str | None = None
SQS_POLLING_TIMEOUT_SECONDS: int = 60
CELERY_BEAT_POLL_INTERVAL: int = (
0 # 0 = use individual defaults; set e.g. 300 for 5-min polling
)
# Daily.co integration
DAILY_API_KEY: str | None = None

View File

@@ -1,80 +0,0 @@
"""Create or update an admin user with password authentication.
Usage:
uv run python -m reflector.tools.create_admin --email admin@localhost --password <pass>
uv run python -m reflector.tools.create_admin --email admin@localhost # prompts for password
uv run python -m reflector.tools.create_admin --hash-only --password <pass> # print hash only
"""
import argparse
import asyncio
import getpass
import sys
from reflector.auth.password_utils import hash_password
from reflector.db.users import user_controller
from reflector.utils import generate_uuid4
async def create_admin(email: str, password: str) -> None:
from reflector.db import get_database
database = get_database()
await database.connect()
try:
password_hash = hash_password(password)
existing = await user_controller.get_by_email(email)
if existing:
await user_controller.set_password_hash(existing.id, password_hash)
print(f"Updated password for existing user: {email} (id={existing.id})")
else:
user = await user_controller.create_or_update(
id=generate_uuid4(),
authentik_uid=f"local:{email}",
email=email,
password_hash=password_hash,
)
print(f"Created admin user: {email} (id={user.id})")
finally:
await database.disconnect()
def main():
parser = argparse.ArgumentParser(description="Create or update an admin user")
parser.add_argument(
"--email", default="admin@localhost", help="Admin email address"
)
parser.add_argument(
"--password",
help="Admin password (will prompt if not provided)",
)
parser.add_argument(
"--hash-only",
action="store_true",
help="Print the password hash and exit (for ADMIN_PASSWORD_HASH env var)",
)
args = parser.parse_args()
password = args.password
if not password:
password = getpass.getpass("Password: ")
confirm = getpass.getpass("Confirm password: ")
if password != confirm:
print("Passwords do not match", file=sys.stderr)
sys.exit(1)
if not password:
print("Password cannot be empty", file=sys.stderr)
sys.exit(1)
if args.hash_only:
print(hash_password(password))
sys.exit(0)
asyncio.run(create_admin(args.email, password))
if __name__ == "__main__":
main()

View File

@@ -24,9 +24,6 @@ from reflector.pipelines.main_live_pipeline import (
pipeline_process as live_pipeline_process,
)
from reflector.storage import Storage
from reflector.worker.app import (
app as celery_app, # noqa: F401 - ensure Celery uses Redis broker
)
def validate_s3_bucket_name(bucket: str) -> None:

View File

@@ -1,43 +0,0 @@
"""Provision admin user on server startup using environment variables.
Reads ADMIN_EMAIL and ADMIN_PASSWORD_HASH from settings and creates or updates
the admin user. Intended to be called from runserver.sh on container startup.
"""
import asyncio
from reflector.db.users import user_controller
from reflector.settings import settings
from reflector.utils import generate_uuid4
async def provision() -> None:
if not settings.ADMIN_EMAIL or not settings.ADMIN_PASSWORD_HASH:
return
from reflector.db import get_database
database = get_database()
await database.connect()
try:
existing = await user_controller.get_by_email(settings.ADMIN_EMAIL)
if existing:
await user_controller.set_password_hash(
existing.id, settings.ADMIN_PASSWORD_HASH
)
print(f"Updated admin user: {settings.ADMIN_EMAIL}")
else:
await user_controller.create_or_update(
id=generate_uuid4(),
authentik_uid=f"local:{settings.ADMIN_EMAIL}",
email=settings.ADMIN_EMAIL,
password_hash=settings.ADMIN_PASSWORD_HASH,
)
print(f"Created admin user: {settings.ADMIN_EMAIL}")
finally:
await database.disconnect()
if __name__ == "__main__":
asyncio.run(provision())

View File

@@ -10,7 +10,6 @@ from pydantic import BaseModel
from reflector.events import subscribers_shutdown
from reflector.logger import logger
from reflector.pipelines.runner import PipelineRunner
from reflector.settings import settings
sessions = []
router = APIRouter()
@@ -124,16 +123,7 @@ async def rtc_offer_base(
# update metrics
m_rtc_sessions.inc()
sdp = pc.localDescription.sdp
# Rewrite ICE candidate IPs when running behind Docker bridge networking
if settings.WEBRTC_HOST:
from reflector.webrtc_ports import resolve_webrtc_host, rewrite_sdp_host
host_ip = resolve_webrtc_host(settings.WEBRTC_HOST)
sdp = rewrite_sdp_host(sdp, host_ip)
return RtcOffer(sdp=sdp, type=pc.localDescription.type)
return RtcOffer(sdp=pc.localDescription.sdp, type=pc.localDescription.type)
@subscribers_shutdown.append

View File

@@ -111,7 +111,6 @@ class GetTranscriptMinimal(BaseModel):
room_id: str | None = None
room_name: str | None = None
audio_deleted: bool | None = None
change_seq: int | None = None
class TranscriptParticipantWithEmail(TranscriptParticipant):
@@ -267,22 +266,12 @@ async def transcripts_list(
source_kind: SourceKind | None = None,
room_id: str | None = None,
search_term: str | None = None,
change_seq_from: int | None = None,
sort_by: Literal["created_at", "change_seq"] | None = None,
):
if not user and not settings.PUBLIC_MODE:
raise HTTPException(status_code=401, detail="Not authenticated")
user_id = user["sub"] if user else None
# Default behavior preserved: sort_by=None → "-created_at"
if sort_by == "change_seq":
order_by = "change_seq" # ASC (ascending for checkpoint-based polling)
elif sort_by == "created_at":
order_by = "-created_at" # DESC (newest first, same as current default)
else:
order_by = "-created_at" # default, backward compatible
return await apaginate(
get_database(),
await transcripts_controller.get_all(
@@ -290,8 +279,7 @@ async def transcripts_list(
source_kind=SourceKind(source_kind) if source_kind else None,
room_id=room_id,
search_term=search_term,
order_by=order_by,
change_seq_from=change_seq_from,
order_by="-created_at",
return_query=True,
),
)
@@ -524,7 +512,6 @@ async def transcript_get(
"room_id": transcript.room_id,
"room_name": room_name,
"audio_deleted": transcript.audio_deleted,
"change_seq": transcript.change_seq,
"participants": participants,
}

View File

@@ -4,22 +4,18 @@ Transcripts websocket API
"""
from fastapi import APIRouter, HTTPException, WebSocket, WebSocketDisconnect
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, WebSocket, WebSocketDisconnect
import reflector.auth as auth
from reflector.db.transcripts import transcripts_controller
from reflector.ws_events import TranscriptWsEvent
from reflector.ws_manager import get_ws_manager
router = APIRouter()
@router.get(
"/transcripts/{transcript_id}/events",
response_model=TranscriptWsEvent,
summary="Transcript WebSocket event schema",
description="Stub exposing the discriminated union of all transcript-level WS events for OpenAPI type generation. Real events are delivered over the WebSocket at the same path.",
)
@router.get("/transcripts/{transcript_id}/events")
async def transcript_get_websocket_events(transcript_id: str):
pass
@@ -28,9 +24,8 @@ async def transcript_get_websocket_events(transcript_id: str):
async def transcript_events_websocket(
transcript_id: str,
websocket: WebSocket,
user: Optional[auth.UserInfo] = Depends(auth.current_user_optional),
):
_, negotiated_subprotocol = auth.parse_ws_bearer_token(websocket)
user = await auth.current_user_ws_optional(websocket)
user_id = user["sub"] if user else None
transcript = await transcripts_controller.get_by_id_for_http(
transcript_id, user_id=user_id
@@ -42,9 +37,7 @@ async def transcript_events_websocket(
# use ts:transcript_id as room id
room_id = f"ts:{transcript_id}"
ws_manager = get_ws_manager()
await ws_manager.add_user_to_room(
room_id, websocket, subprotocol=negotiated_subprotocol
)
await ws_manager.add_user_to_room(room_id, websocket)
try:
# on first connection, send all events only to the current user

View File

@@ -2,47 +2,54 @@ from typing import Optional
from fastapi import APIRouter, WebSocket, WebSocketDisconnect
import reflector.auth as auth
from reflector.ws_events import UserWsEvent
from reflector.auth.auth_jwt import JWTAuth # type: ignore
from reflector.db.users import user_controller
from reflector.ws_manager import get_ws_manager
router = APIRouter()
@router.get(
"/events",
response_model=UserWsEvent,
summary="User WebSocket event schema",
description="Stub exposing the discriminated union of all user-level WS events for OpenAPI type generation. Real events are delivered over the WebSocket at the same path.",
)
async def user_get_websocket_events():
pass
# Close code for unauthorized WebSocket connections
UNAUTHORISED = 4401
@router.websocket("/events")
async def user_events_websocket(websocket: WebSocket):
token, negotiated_subprotocol = auth.parse_ws_bearer_token(websocket)
# Browser can't send Authorization header for WS; use subprotocol: ["bearer", token]
raw_subprotocol = websocket.headers.get("sec-websocket-protocol") or ""
parts = [p.strip() for p in raw_subprotocol.split(",") if p.strip()]
token: Optional[str] = None
negotiated_subprotocol: Optional[str] = None
if len(parts) >= 2 and parts[0].lower() == "bearer":
negotiated_subprotocol = "bearer"
token = parts[1]
user_id: Optional[str] = None
if not token:
await websocket.close(code=UNAUTHORISED)
return
try:
user = await auth.current_user_ws_optional(websocket)
payload = JWTAuth().verify_token(token)
authentik_uid = payload.get("sub")
if authentik_uid:
user = await user_controller.get_by_authentik_uid(authentik_uid)
if user:
user_id = user.id
else:
await websocket.close(code=UNAUTHORISED)
return
else:
await websocket.close(code=UNAUTHORISED)
return
except Exception:
await websocket.close(code=UNAUTHORISED)
return
if not user:
if not user_id:
await websocket.close(code=UNAUTHORISED)
return
user_id: Optional[str] = user.sub if hasattr(user, "sub") else user["sub"]
room_id = f"user:{user_id}"
ws_manager = get_ws_manager()

View File

@@ -1,111 +0,0 @@
"""
Monkey-patch aioice to use a fixed UDP port range for ICE candidates,
and optionally rewrite SDP to advertise a different host IP.
This allows running the server in Docker with bridge networking
(no network_mode: host) by:
1. Restricting ICE UDP ports to a known range that can be mapped in Docker
2. Replacing container-internal IPs with the Docker host IP in SDP answers
"""
import asyncio
import socket
from reflector.logger import logger
def parse_port_range(range_str: str) -> tuple[int, int]:
"""Parse a 'min-max' string into (min_port, max_port)."""
parts = range_str.split("-")
if len(parts) != 2:
raise ValueError(f"WEBRTC_PORT_RANGE must be 'min-max', got: {range_str!r}")
min_port, max_port = int(parts[0]), int(parts[1])
if not (1024 <= min_port <= max_port <= 65535):
raise ValueError(
f"Invalid port range: {min_port}-{max_port} "
"(must be 1024-65535 with min <= max)"
)
return min_port, max_port
def patch_aioice_port_range(min_port: int, max_port: int) -> None:
"""
Monkey-patch aioice so that ICE candidate UDP sockets bind to ports
within [min_port, max_port] instead of OS-assigned ephemeral ports.
Works by temporarily wrapping loop.create_datagram_endpoint() during
aioice's get_component_candidates() to intercept bind(addr, 0) calls.
"""
import aioice.ice as _ice
_original = _ice.Connection.get_component_candidates
_state = {"next_port": min_port}
async def _patched_get_component_candidates(self, component, addresses, timeout=5):
loop = asyncio.get_event_loop()
_orig_create = loop.create_datagram_endpoint
async def _create_with_port_range(*args, **kwargs):
local_addr = kwargs.get("local_addr")
if local_addr and local_addr[1] == 0:
addr = local_addr[0]
# Try each port in the range (wrapping around)
attempts = max_port - min_port + 1
for _ in range(attempts):
port = _state["next_port"]
_state["next_port"] = (
min_port
if _state["next_port"] >= max_port
else _state["next_port"] + 1
)
try:
kwargs["local_addr"] = (addr, port)
return await _orig_create(*args, **kwargs)
except OSError:
continue
# All ports exhausted, fall back to OS assignment
logger.warning(
"All WebRTC ports in range exhausted, falling back to OS",
min_port=min_port,
max_port=max_port,
)
kwargs["local_addr"] = (addr, 0)
return await _orig_create(*args, **kwargs)
loop.create_datagram_endpoint = _create_with_port_range
try:
return await _original(self, component, addresses, timeout)
finally:
loop.create_datagram_endpoint = _orig_create
_ice.Connection.get_component_candidates = _patched_get_component_candidates
logger.info(
"aioice patched for WebRTC port range",
min_port=min_port,
max_port=max_port,
)
def resolve_webrtc_host(host: str) -> str:
"""Resolve a hostname or IP to an IP address for ICE candidate rewriting."""
try:
ip = socket.gethostbyname(host)
logger.info("Resolved WEBRTC_HOST", host=host, ip=ip)
return ip
except socket.gaierror:
logger.warning("Could not resolve WEBRTC_HOST, using as-is", host=host)
return host
def rewrite_sdp_host(sdp: str, target_ip: str) -> str:
"""
Replace container-internal IPs in SDP with target_ip so that
ICE candidates advertise a routable address.
"""
import aioice.ice
container_ips = aioice.ice.get_host_addresses(use_ipv4=True, use_ipv6=False)
for ip in container_ips:
if ip != "127.0.0.1" and ip != target_ip:
sdp = sdp.replace(ip, target_ip)
return sdp

View File

@@ -8,21 +8,8 @@ from reflector.settings import settings
logger = structlog.get_logger(__name__)
# Polling intervals (seconds)
# CELERY_BEAT_POLL_INTERVAL overrides all sub-5-min intervals (e.g. 300 for selfhosted)
_override = (
float(settings.CELERY_BEAT_POLL_INTERVAL)
if settings.CELERY_BEAT_POLL_INTERVAL > 0
else 0
)
# Webhook-aware: 180s when webhook configured (backup mode), 15s when no webhook (primary discovery)
POLL_DAILY_RECORDINGS_INTERVAL_SEC = _override or (
180.0 if settings.DAILY_WEBHOOK_SECRET else 15.0
)
SQS_POLL_INTERVAL = _override or float(settings.SQS_POLLING_TIMEOUT_SECONDS)
RECONCILIATION_INTERVAL = _override or 30.0
ICS_SYNC_INTERVAL = _override or 60.0
UPCOMING_MEETINGS_INTERVAL = _override or 30.0
POLL_DAILY_RECORDINGS_INTERVAL_SEC = 180.0 if settings.DAILY_WEBHOOK_SECRET else 15.0
if celery.current_app.main != "default":
logger.info(f"Celery already configured ({celery.current_app})")
@@ -46,11 +33,11 @@ else:
app.conf.beat_schedule = {
"process_messages": {
"task": "reflector.worker.process.process_messages",
"schedule": SQS_POLL_INTERVAL,
"schedule": float(settings.SQS_POLLING_TIMEOUT_SECONDS),
},
"process_meetings": {
"task": "reflector.worker.process.process_meetings",
"schedule": SQS_POLL_INTERVAL,
"schedule": float(settings.SQS_POLLING_TIMEOUT_SECONDS),
},
"reprocess_failed_recordings": {
"task": "reflector.worker.process.reprocess_failed_recordings",
@@ -66,15 +53,15 @@ else:
},
"trigger_daily_reconciliation": {
"task": "reflector.worker.process.trigger_daily_reconciliation",
"schedule": RECONCILIATION_INTERVAL,
"schedule": 30.0, # Every 30 seconds (queues poll tasks for all active meetings)
},
"sync_all_ics_calendars": {
"task": "reflector.worker.ics_sync.sync_all_ics_calendars",
"schedule": ICS_SYNC_INTERVAL,
"schedule": 60.0, # Run every minute to check which rooms need sync
},
"create_upcoming_meetings": {
"task": "reflector.worker.ics_sync.create_upcoming_meetings",
"schedule": UPCOMING_MEETINGS_INTERVAL,
"schedule": 30.0, # Run every 30 seconds to create upcoming meetings
},
}

View File

@@ -1,188 +0,0 @@
"""Typed WebSocket event models.
Defines Pydantic models with Literal discriminators for all WS events.
Exposed via stub GET endpoints so ``pnpm openapi`` generates TS discriminated unions.
"""
from typing import Annotated, Literal, Union
from pydantic import BaseModel, Discriminator
from reflector.db.transcripts import (
TranscriptActionItems,
TranscriptDuration,
TranscriptFinalLongSummary,
TranscriptFinalShortSummary,
TranscriptFinalTitle,
TranscriptStatus,
TranscriptText,
TranscriptWaveform,
)
from reflector.utils.string import NonEmptyString
from reflector.views.transcripts import GetTranscriptTopic
# ---------------------------------------------------------------------------
# Transcript-level event name literal
# ---------------------------------------------------------------------------
TranscriptEventName = Literal[
"TRANSCRIPT",
"TOPIC",
"STATUS",
"FINAL_TITLE",
"FINAL_LONG_SUMMARY",
"FINAL_SHORT_SUMMARY",
"ACTION_ITEMS",
"DURATION",
"WAVEFORM",
]
# ---------------------------------------------------------------------------
# Transcript-level WS event wrappers
# ---------------------------------------------------------------------------
class TranscriptWsTranscript(BaseModel):
event: Literal["TRANSCRIPT"] = "TRANSCRIPT"
data: TranscriptText
class TranscriptWsTopic(BaseModel):
event: Literal["TOPIC"] = "TOPIC"
data: GetTranscriptTopic
class TranscriptWsStatusData(BaseModel):
value: TranscriptStatus
class TranscriptWsStatus(BaseModel):
event: Literal["STATUS"] = "STATUS"
data: TranscriptWsStatusData
class TranscriptWsFinalTitle(BaseModel):
event: Literal["FINAL_TITLE"] = "FINAL_TITLE"
data: TranscriptFinalTitle
class TranscriptWsFinalLongSummary(BaseModel):
event: Literal["FINAL_LONG_SUMMARY"] = "FINAL_LONG_SUMMARY"
data: TranscriptFinalLongSummary
class TranscriptWsFinalShortSummary(BaseModel):
event: Literal["FINAL_SHORT_SUMMARY"] = "FINAL_SHORT_SUMMARY"
data: TranscriptFinalShortSummary
class TranscriptWsActionItems(BaseModel):
event: Literal["ACTION_ITEMS"] = "ACTION_ITEMS"
data: TranscriptActionItems
class TranscriptWsDuration(BaseModel):
event: Literal["DURATION"] = "DURATION"
data: TranscriptDuration
class TranscriptWsWaveform(BaseModel):
event: Literal["WAVEFORM"] = "WAVEFORM"
data: TranscriptWaveform
TranscriptWsEvent = Annotated[
Union[
TranscriptWsTranscript,
TranscriptWsTopic,
TranscriptWsStatus,
TranscriptWsFinalTitle,
TranscriptWsFinalLongSummary,
TranscriptWsFinalShortSummary,
TranscriptWsActionItems,
TranscriptWsDuration,
TranscriptWsWaveform,
],
Discriminator("event"),
]
# ---------------------------------------------------------------------------
# User-level event name literal
# ---------------------------------------------------------------------------
UserEventName = Literal[
"TRANSCRIPT_CREATED",
"TRANSCRIPT_DELETED",
"TRANSCRIPT_STATUS",
"TRANSCRIPT_FINAL_TITLE",
"TRANSCRIPT_DURATION",
]
# ---------------------------------------------------------------------------
# User-level WS event data models
# ---------------------------------------------------------------------------
class UserTranscriptCreatedData(BaseModel):
id: NonEmptyString
class UserTranscriptDeletedData(BaseModel):
id: NonEmptyString
class UserTranscriptStatusData(BaseModel):
id: NonEmptyString
value: TranscriptStatus
class UserTranscriptFinalTitleData(BaseModel):
id: NonEmptyString
title: NonEmptyString
class UserTranscriptDurationData(BaseModel):
id: NonEmptyString
duration: float
# ---------------------------------------------------------------------------
# User-level WS event wrappers
# ---------------------------------------------------------------------------
class UserWsTranscriptCreated(BaseModel):
event: Literal["TRANSCRIPT_CREATED"] = "TRANSCRIPT_CREATED"
data: UserTranscriptCreatedData
class UserWsTranscriptDeleted(BaseModel):
event: Literal["TRANSCRIPT_DELETED"] = "TRANSCRIPT_DELETED"
data: UserTranscriptDeletedData
class UserWsTranscriptStatus(BaseModel):
event: Literal["TRANSCRIPT_STATUS"] = "TRANSCRIPT_STATUS"
data: UserTranscriptStatusData
class UserWsTranscriptFinalTitle(BaseModel):
event: Literal["TRANSCRIPT_FINAL_TITLE"] = "TRANSCRIPT_FINAL_TITLE"
data: UserTranscriptFinalTitleData
class UserWsTranscriptDuration(BaseModel):
event: Literal["TRANSCRIPT_DURATION"] = "TRANSCRIPT_DURATION"
data: UserTranscriptDurationData
UserWsEvent = Annotated[
Union[
UserWsTranscriptCreated,
UserWsTranscriptDeleted,
UserWsTranscriptStatus,
UserWsTranscriptFinalTitle,
UserWsTranscriptDuration,
],
Discriminator("event"),
]

View File

@@ -2,10 +2,6 @@
if [ "${ENTRYPOINT}" = "server" ]; then
uv run alembic upgrade head
# Provision admin user if password auth is configured
if [ -n "${ADMIN_EMAIL:-}" ] && [ -n "${ADMIN_PASSWORD_HASH:-}" ]; then
uv run python -m reflector.tools.provision_admin
fi
uv run uvicorn reflector.app:app --host 0.0.0.0 --port 1250
elif [ "${ENTRYPOINT}" = "worker" ]; then
uv run celery -A reflector.worker.app worker --loglevel=info

View File

@@ -1,201 +0,0 @@
"""Tests for the password auth backend."""
import pytest
from httpx import AsyncClient
from jose import jwt
from reflector.auth.password_utils import hash_password
from reflector.settings import settings
@pytest.fixture
async def password_app():
"""Create a minimal FastAPI app with the password auth router."""
from fastapi import FastAPI
from reflector.auth import auth_password
app = FastAPI()
app.include_router(auth_password.router, prefix="/v1")
# Reset rate limiter between tests
auth_password._login_attempts.clear()
return app
@pytest.fixture
async def password_client(password_app):
"""Create a test client for the password auth app."""
async with AsyncClient(app=password_app, base_url="http://test/v1") as client:
yield client
async def _create_user_with_password(email: str, password: str):
"""Helper to create a user with a password hash in the DB."""
from reflector.db.users import user_controller
from reflector.utils import generate_uuid4
pw_hash = hash_password(password)
return await user_controller.create_or_update(
id=generate_uuid4(),
authentik_uid=f"local:{email}",
email=email,
password_hash=pw_hash,
)
@pytest.mark.asyncio
async def test_login_success(password_client, setup_database):
await _create_user_with_password("admin@test.com", "testpass123")
response = await password_client.post(
"/auth/login",
json={"email": "admin@test.com", "password": "testpass123"},
)
assert response.status_code == 200
data = response.json()
assert "access_token" in data
assert data["token_type"] == "bearer"
assert data["expires_in"] > 0
# Verify the JWT is valid
payload = jwt.decode(
data["access_token"],
settings.SECRET_KEY,
algorithms=["HS256"],
)
assert payload["email"] == "admin@test.com"
assert "sub" in payload
assert "exp" in payload
@pytest.mark.asyncio
async def test_login_wrong_password(password_client, setup_database):
await _create_user_with_password("user@test.com", "correctpassword")
response = await password_client.post(
"/auth/login",
json={"email": "user@test.com", "password": "wrongpassword"},
)
assert response.status_code == 401
@pytest.mark.asyncio
async def test_login_nonexistent_user(password_client, setup_database):
response = await password_client.post(
"/auth/login",
json={"email": "nobody@test.com", "password": "anything"},
)
assert response.status_code == 401
@pytest.mark.asyncio
async def test_login_user_without_password_hash(password_client, setup_database):
"""User exists but has no password_hash (e.g. Authentik user)."""
from reflector.db.users import user_controller
from reflector.utils import generate_uuid4
await user_controller.create_or_update(
id=generate_uuid4(),
authentik_uid="authentik:abc123",
email="oidc@test.com",
)
response = await password_client.post(
"/auth/login",
json={"email": "oidc@test.com", "password": "anything"},
)
assert response.status_code == 401
@pytest.mark.asyncio
async def test_login_rate_limiting(password_client, setup_database):
from reflector.auth import auth_password
# Reset rate limiter
auth_password._login_attempts.clear()
for _ in range(10):
await password_client.post(
"/auth/login",
json={"email": "fake@test.com", "password": "wrong"},
)
# 11th attempt should be rate-limited
response = await password_client.post(
"/auth/login",
json={"email": "fake@test.com", "password": "wrong"},
)
assert response.status_code == 429
@pytest.mark.asyncio
async def test_jwt_create_and_verify():
from reflector.auth.auth_password import _create_access_token, _verify_token
token, expires_in = _create_access_token("user-123", "test@example.com")
assert expires_in > 0
payload = _verify_token(token)
assert payload["sub"] == "user-123"
assert payload["email"] == "test@example.com"
assert "exp" in payload
@pytest.mark.asyncio
async def test_authenticate_user_with_jwt():
from reflector.auth.auth_password import (
_authenticate_user,
_create_access_token,
)
token, _ = _create_access_token("user-abc", "abc@test.com")
user = await _authenticate_user(token, None)
assert user is not None
assert user.sub == "user-abc"
assert user.email == "abc@test.com"
@pytest.mark.asyncio
async def test_authenticate_user_invalid_jwt():
from fastapi import HTTPException
from reflector.auth.auth_password import _authenticate_user
with pytest.raises(HTTPException) as exc_info:
await _authenticate_user("invalid.jwt.token", None)
assert exc_info.value.status_code == 401
@pytest.mark.asyncio
async def test_authenticate_user_no_credentials():
from reflector.auth.auth_password import _authenticate_user
user = await _authenticate_user(None, None)
assert user is None
@pytest.mark.asyncio
async def test_current_user_raises_without_token():
"""Verify that current_user dependency raises 401 without token."""
from fastapi import Depends, FastAPI
from fastapi.testclient import TestClient
from reflector.auth import auth_password
app = FastAPI()
@app.get("/test")
async def test_endpoint(user=Depends(auth_password.current_user)):
return {"user": user.sub}
# Use sync TestClient for simplicity
client = TestClient(app)
response = client.get("/test")
# OAuth2PasswordBearer with auto_error=False returns None, then current_user raises 401
assert response.status_code == 401

View File

@@ -1,97 +0,0 @@
"""Tests for admin user creation logic (used by create_admin CLI tool)."""
import pytest
from reflector.auth.password_utils import hash_password, verify_password
from reflector.db.users import user_controller
from reflector.utils import generate_uuid4
async def _provision_admin(email: str, password: str):
"""Mirrors the logic in create_admin.create_admin() without managing DB connections."""
password_hash = hash_password(password)
existing = await user_controller.get_by_email(email)
if existing:
await user_controller.set_password_hash(existing.id, password_hash)
else:
await user_controller.create_or_update(
id=generate_uuid4(),
authentik_uid=f"local:{email}",
email=email,
password_hash=password_hash,
)
@pytest.mark.asyncio
async def test_create_admin_new_user(setup_database):
await _provision_admin("newadmin@test.com", "password123")
user = await user_controller.get_by_email("newadmin@test.com")
assert user is not None
assert user.email == "newadmin@test.com"
assert user.authentik_uid == "local:newadmin@test.com"
assert user.password_hash is not None
assert verify_password("password123", user.password_hash)
@pytest.mark.asyncio
async def test_create_admin_updates_existing(setup_database):
# Create first
await _provision_admin("admin@test.com", "oldpassword")
user1 = await user_controller.get_by_email("admin@test.com")
# Update password
await _provision_admin("admin@test.com", "newpassword")
user2 = await user_controller.get_by_email("admin@test.com")
assert user1.id == user2.id # same user, not duplicated
assert verify_password("newpassword", user2.password_hash)
assert not verify_password("oldpassword", user2.password_hash)
@pytest.mark.asyncio
async def test_create_admin_idempotent(setup_database):
await _provision_admin("admin@test.com", "samepassword")
await _provision_admin("admin@test.com", "samepassword")
# Should only have one user
users = await user_controller.list_all()
admin_users = [u for u in users if u.email == "admin@test.com"]
assert len(admin_users) == 1
@pytest.mark.asyncio
async def test_create_or_update_with_password_hash(setup_database):
"""Test the extended create_or_update method with password_hash parameter."""
pw_hash = hash_password("test123")
user = await user_controller.create_or_update(
id=generate_uuid4(),
authentik_uid="local:test@example.com",
email="test@example.com",
password_hash=pw_hash,
)
assert user.password_hash == pw_hash
fetched = await user_controller.get_by_email("test@example.com")
assert fetched is not None
assert verify_password("test123", fetched.password_hash)
@pytest.mark.asyncio
async def test_set_password_hash(setup_database):
"""Test the set_password_hash method."""
user = await user_controller.create_or_update(
id=generate_uuid4(),
authentik_uid="local:pw@test.com",
email="pw@test.com",
)
assert user.password_hash is None
pw_hash = hash_password("newpass")
await user_controller.set_password_hash(user.id, pw_hash)
updated = await user_controller.get_by_email("pw@test.com")
assert updated is not None
assert verify_password("newpass", updated.password_hash)

View File

@@ -1,11 +1,13 @@
"""Tests for LLM structured output with astructured_predict + reflection retry"""
"""Tests for LLM parse error recovery using llama-index Workflow"""
from unittest.mock import AsyncMock, patch
from time import monotonic
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from pydantic import BaseModel, Field, ValidationError
from pydantic import BaseModel, Field
from workflows.errors import WorkflowRuntimeError, WorkflowTimeoutError
from reflector.llm import LLM, LLMParseError
from reflector.llm import LLM, LLMParseError, StructuredOutputWorkflow
from reflector.utils.retry import RetryException
@@ -17,43 +19,51 @@ class TestResponse(BaseModel):
confidence: float = Field(description="Confidence score", ge=0, le=1)
def make_completion_response(text: str):
"""Create a mock CompletionResponse with .text attribute"""
response = MagicMock()
response.text = text
return response
class TestLLMParseErrorRecovery:
"""Test parse error recovery with astructured_predict reflection loop"""
"""Test parse error recovery with Workflow feedback loop"""
@pytest.mark.asyncio
async def test_parse_error_recovery_with_feedback(self, test_settings):
"""Test that parse errors trigger retry with reflection prompt"""
"""Test that parse errors trigger retry with error feedback"""
llm = LLM(settings=test_settings, temperature=0.4, max_tokens=100)
call_count = {"count": 0}
async def astructured_predict_handler(output_cls, prompt_tmpl, **kwargs):
call_count["count"] += 1
if call_count["count"] == 1:
# First call: raise ValidationError (missing fields)
raise ValidationError.from_exception_data(
title="TestResponse",
line_errors=[
{
"type": "missing",
"loc": ("summary",),
"msg": "Field required",
"input": {"title": "Test"},
}
],
)
else:
# Second call: should have reflection in the prompt
assert "reflection" in kwargs
assert "could not be parsed" in kwargs["reflection"]
assert "Error:" in kwargs["reflection"]
return TestResponse(title="Test", summary="Summary", confidence=0.95)
with patch("reflector.llm.Settings") as mock_settings:
mock_settings.llm.astructured_predict = AsyncMock(
side_effect=astructured_predict_handler
with (
patch("reflector.llm.TreeSummarize") as mock_summarize,
patch("reflector.llm.Settings") as mock_settings,
):
mock_summarizer = MagicMock()
mock_summarize.return_value = mock_summarizer
# TreeSummarize returns plain text analysis (step 1)
mock_summarizer.aget_response = AsyncMock(
return_value="The analysis shows a test with summary and high confidence."
)
call_count = {"count": 0}
async def acomplete_handler(prompt, *args, **kwargs):
call_count["count"] += 1
if call_count["count"] == 1:
# First JSON formatting call returns invalid JSON
return make_completion_response('{"title": "Test"}')
else:
# Second call should have error feedback in prompt
assert "Your previous response could not be parsed:" in prompt
assert '{"title": "Test"}' in prompt
assert "Error:" in prompt
assert "Please try again" in prompt
return make_completion_response(
'{"title": "Test", "summary": "Summary", "confidence": 0.95}'
)
mock_settings.llm.acomplete = AsyncMock(side_effect=acomplete_handler)
result = await llm.get_structured_response(
prompt="Test prompt", texts=["Test text"], output_cls=TestResponse
)
@@ -61,6 +71,8 @@ class TestLLMParseErrorRecovery:
assert result.title == "Test"
assert result.summary == "Summary"
assert result.confidence == 0.95
# TreeSummarize called once, Settings.llm.acomplete called twice
assert mock_summarizer.aget_response.call_count == 1
assert call_count["count"] == 2
@pytest.mark.asyncio
@@ -68,22 +80,20 @@ class TestLLMParseErrorRecovery:
"""Test that parse error retry stops after max attempts"""
llm = LLM(settings=test_settings, temperature=0.4, max_tokens=100)
# Always raise ValidationError
async def always_fail(output_cls, prompt_tmpl, **kwargs):
raise ValidationError.from_exception_data(
title="TestResponse",
line_errors=[
{
"type": "missing",
"loc": ("summary",),
"msg": "Field required",
"input": {},
}
],
)
with (
patch("reflector.llm.TreeSummarize") as mock_summarize,
patch("reflector.llm.Settings") as mock_settings,
):
mock_summarizer = MagicMock()
mock_summarize.return_value = mock_summarizer
mock_summarizer.aget_response = AsyncMock(return_value="Some analysis")
with patch("reflector.llm.Settings") as mock_settings:
mock_settings.llm.astructured_predict = AsyncMock(side_effect=always_fail)
# Always return invalid JSON from acomplete
mock_settings.llm.acomplete = AsyncMock(
return_value=make_completion_response(
'{"invalid": "missing required fields"}'
)
)
with pytest.raises(LLMParseError, match="Failed to parse"):
await llm.get_structured_response(
@@ -91,38 +101,35 @@ class TestLLMParseErrorRecovery:
)
expected_attempts = test_settings.LLM_PARSE_MAX_RETRIES + 1
assert mock_settings.llm.astructured_predict.call_count == expected_attempts
# TreeSummarize called once, acomplete called max_retries times
assert mock_summarizer.aget_response.call_count == 1
assert mock_settings.llm.acomplete.call_count == expected_attempts
@pytest.mark.asyncio
async def test_raw_response_logging_on_parse_error(self, test_settings, caplog):
"""Test that raw response is logged when parse error occurs"""
llm = LLM(settings=test_settings, temperature=0.4, max_tokens=100)
call_count = {"count": 0}
async def astructured_predict_handler(output_cls, prompt_tmpl, **kwargs):
call_count["count"] += 1
if call_count["count"] == 1:
raise ValidationError.from_exception_data(
title="TestResponse",
line_errors=[
{
"type": "missing",
"loc": ("summary",),
"msg": "Field required",
"input": {"title": "Test"},
}
],
)
return TestResponse(title="Test", summary="Summary", confidence=0.95)
with (
patch("reflector.llm.TreeSummarize") as mock_summarize,
patch("reflector.llm.Settings") as mock_settings,
caplog.at_level("ERROR"),
):
mock_settings.llm.astructured_predict = AsyncMock(
side_effect=astructured_predict_handler
)
mock_summarizer = MagicMock()
mock_summarize.return_value = mock_summarizer
mock_summarizer.aget_response = AsyncMock(return_value="Some analysis")
call_count = {"count": 0}
async def acomplete_handler(*args, **kwargs):
call_count["count"] += 1
if call_count["count"] == 1:
return make_completion_response('{"title": "Test"}') # Invalid
return make_completion_response(
'{"title": "Test", "summary": "Summary", "confidence": 0.95}'
)
mock_settings.llm.acomplete = AsyncMock(side_effect=acomplete_handler)
result = await llm.get_structured_response(
prompt="Test prompt", texts=["Test text"], output_cls=TestResponse
@@ -136,45 +143,35 @@ class TestLLMParseErrorRecovery:
@pytest.mark.asyncio
async def test_multiple_validation_errors_in_feedback(self, test_settings):
"""Test that validation errors are included in reflection feedback"""
"""Test that validation errors are included in feedback"""
llm = LLM(settings=test_settings, temperature=0.4, max_tokens=100)
call_count = {"count": 0}
with (
patch("reflector.llm.TreeSummarize") as mock_summarize,
patch("reflector.llm.Settings") as mock_settings,
):
mock_summarizer = MagicMock()
mock_summarize.return_value = mock_summarizer
mock_summarizer.aget_response = AsyncMock(return_value="Some analysis")
async def astructured_predict_handler(output_cls, prompt_tmpl, **kwargs):
call_count["count"] += 1
if call_count["count"] == 1:
# Missing title and summary
raise ValidationError.from_exception_data(
title="TestResponse",
line_errors=[
{
"type": "missing",
"loc": ("title",),
"msg": "Field required",
"input": {},
},
{
"type": "missing",
"loc": ("summary",),
"msg": "Field required",
"input": {},
},
],
)
else:
# Should have schema validation errors in reflection
assert "reflection" in kwargs
assert (
"Schema validation errors" in kwargs["reflection"]
or "error" in kwargs["reflection"].lower()
)
return TestResponse(title="Test", summary="Summary", confidence=0.95)
call_count = {"count": 0}
with patch("reflector.llm.Settings") as mock_settings:
mock_settings.llm.astructured_predict = AsyncMock(
side_effect=astructured_predict_handler
)
async def acomplete_handler(prompt, *args, **kwargs):
call_count["count"] += 1
if call_count["count"] == 1:
# Missing title and summary
return make_completion_response('{"confidence": 0.5}')
else:
# Should have schema validation errors in prompt
assert (
"Schema validation errors" in prompt
or "error" in prompt.lower()
)
return make_completion_response(
'{"title": "Test", "summary": "Summary", "confidence": 0.95}'
)
mock_settings.llm.acomplete = AsyncMock(side_effect=acomplete_handler)
result = await llm.get_structured_response(
prompt="Test prompt", texts=["Test text"], output_cls=TestResponse
@@ -188,10 +185,17 @@ class TestLLMParseErrorRecovery:
"""Test that no retry happens when first attempt succeeds"""
llm = LLM(settings=test_settings, temperature=0.4, max_tokens=100)
with patch("reflector.llm.Settings") as mock_settings:
mock_settings.llm.astructured_predict = AsyncMock(
return_value=TestResponse(
title="Test", summary="Summary", confidence=0.95
with (
patch("reflector.llm.TreeSummarize") as mock_summarize,
patch("reflector.llm.Settings") as mock_settings,
):
mock_summarizer = MagicMock()
mock_summarize.return_value = mock_summarizer
mock_summarizer.aget_response = AsyncMock(return_value="Some analysis")
mock_settings.llm.acomplete = AsyncMock(
return_value=make_completion_response(
'{"title": "Test", "summary": "Summary", "confidence": 0.95}'
)
)
@@ -202,28 +206,195 @@ class TestLLMParseErrorRecovery:
assert result.title == "Test"
assert result.summary == "Summary"
assert result.confidence == 0.95
assert mock_settings.llm.astructured_predict.call_count == 1
assert mock_summarizer.aget_response.call_count == 1
assert mock_settings.llm.acomplete.call_count == 1
class TestStructuredOutputWorkflow:
"""Direct tests for the StructuredOutputWorkflow"""
@pytest.mark.asyncio
async def test_workflow_retries_on_validation_error(self):
"""Test workflow retries when validation fails"""
workflow = StructuredOutputWorkflow(
output_cls=TestResponse,
max_retries=3,
timeout=30,
)
with (
patch("reflector.llm.TreeSummarize") as mock_summarize,
patch("reflector.llm.Settings") as mock_settings,
):
mock_summarizer = MagicMock()
mock_summarize.return_value = mock_summarizer
mock_summarizer.aget_response = AsyncMock(return_value="Some analysis")
call_count = {"count": 0}
async def acomplete_handler(*args, **kwargs):
call_count["count"] += 1
if call_count["count"] < 2:
return make_completion_response('{"title": "Only title"}')
return make_completion_response(
'{"title": "Test", "summary": "Summary", "confidence": 0.9}'
)
mock_settings.llm.acomplete = AsyncMock(side_effect=acomplete_handler)
result = await workflow.run(
prompt="Extract data",
texts=["Some text"],
tone_name=None,
)
assert "success" in result
assert result["success"].title == "Test"
assert call_count["count"] == 2
@pytest.mark.asyncio
async def test_workflow_returns_error_after_max_retries(self):
"""Test workflow returns error after exhausting retries"""
workflow = StructuredOutputWorkflow(
output_cls=TestResponse,
max_retries=2,
timeout=30,
)
with (
patch("reflector.llm.TreeSummarize") as mock_summarize,
patch("reflector.llm.Settings") as mock_settings,
):
mock_summarizer = MagicMock()
mock_summarize.return_value = mock_summarizer
mock_summarizer.aget_response = AsyncMock(return_value="Some analysis")
# Always return invalid JSON
mock_settings.llm.acomplete = AsyncMock(
return_value=make_completion_response('{"invalid": true}')
)
result = await workflow.run(
prompt="Extract data",
texts=["Some text"],
tone_name=None,
)
assert "error" in result
# TreeSummarize called once, acomplete called max_retries times
assert mock_summarizer.aget_response.call_count == 1
assert mock_settings.llm.acomplete.call_count == 2
class TestNetworkErrorRetries:
"""Test that network errors are retried by the outer retry() wrapper"""
"""Test that network error retries are handled by OpenAILike, not Workflow"""
@pytest.mark.asyncio
async def test_network_error_retried_by_outer_wrapper(self, test_settings):
"""Test that network errors trigger the outer retry wrapper"""
async def test_network_error_propagates_after_openai_retries(self, test_settings):
"""Test that network errors are retried by OpenAILike and then propagate.
Network retries are handled by OpenAILike (max_retries=3), not by our
StructuredOutputWorkflow. This test verifies that network errors propagate
up after OpenAILike exhausts its retries.
"""
llm = LLM(settings=test_settings, temperature=0.4, max_tokens=100)
with (
patch("reflector.llm.TreeSummarize") as mock_summarize,
patch("reflector.llm.Settings") as mock_settings,
):
mock_summarizer = MagicMock()
mock_summarize.return_value = mock_summarizer
mock_summarizer.aget_response = AsyncMock(return_value="Some analysis")
# Simulate network error from acomplete (after OpenAILike retries exhausted)
network_error = ConnectionError("Connection refused")
mock_settings.llm.acomplete = AsyncMock(side_effect=network_error)
# Network error wrapped in WorkflowRuntimeError
with pytest.raises(WorkflowRuntimeError, match="Connection refused"):
await llm.get_structured_response(
prompt="Test prompt", texts=["Test text"], output_cls=TestResponse
)
# acomplete called only once - network error propagates, not retried by Workflow
assert mock_settings.llm.acomplete.call_count == 1
@pytest.mark.asyncio
async def test_network_error_not_retried_by_workflow(self, test_settings):
"""Test that Workflow does NOT retry network errors (OpenAILike handles those).
This verifies the separation of concerns:
- StructuredOutputWorkflow: retries parse/validation errors
- OpenAILike: retries network errors (internally, max_retries=3)
"""
workflow = StructuredOutputWorkflow(
output_cls=TestResponse,
max_retries=3,
timeout=30,
)
with (
patch("reflector.llm.TreeSummarize") as mock_summarize,
patch("reflector.llm.Settings") as mock_settings,
):
mock_summarizer = MagicMock()
mock_summarize.return_value = mock_summarizer
mock_summarizer.aget_response = AsyncMock(return_value="Some analysis")
# Network error should propagate immediately, not trigger Workflow retry
mock_settings.llm.acomplete = AsyncMock(
side_effect=TimeoutError("Request timed out")
)
# Network error wrapped in WorkflowRuntimeError
with pytest.raises(WorkflowRuntimeError, match="Request timed out"):
await workflow.run(
prompt="Extract data",
texts=["Some text"],
tone_name=None,
)
# Only called once - Workflow doesn't retry network errors
assert mock_settings.llm.acomplete.call_count == 1
class TestWorkflowTimeoutRetry:
"""Test timeout retry mechanism in get_structured_response"""
@pytest.mark.asyncio
async def test_timeout_retry_succeeds_on_retry(self, test_settings):
"""Test that WorkflowTimeoutError triggers retry and succeeds"""
llm = LLM(settings=test_settings, temperature=0.4, max_tokens=100)
call_count = {"count": 0}
async def astructured_predict_handler(output_cls, prompt_tmpl, **kwargs):
async def workflow_run_side_effect(*args, **kwargs):
call_count["count"] += 1
if call_count["count"] == 1:
raise ConnectionError("Connection refused")
return TestResponse(title="Test", summary="Summary", confidence=0.95)
raise WorkflowTimeoutError("Operation timed out after 120 seconds")
return {
"success": TestResponse(
title="Test", summary="Summary", confidence=0.95
)
}
with patch("reflector.llm.Settings") as mock_settings:
mock_settings.llm.astructured_predict = AsyncMock(
side_effect=astructured_predict_handler
with (
patch("reflector.llm.StructuredOutputWorkflow") as mock_workflow_class,
patch("reflector.llm.TreeSummarize") as mock_summarize,
patch("reflector.llm.Settings") as mock_settings,
):
mock_workflow = MagicMock()
mock_workflow.run = AsyncMock(side_effect=workflow_run_side_effect)
mock_workflow_class.return_value = mock_workflow
mock_summarizer = MagicMock()
mock_summarize.return_value = mock_summarizer
mock_summarizer.aget_response = AsyncMock(return_value="Some analysis")
mock_settings.llm.acomplete = AsyncMock(
return_value=make_completion_response(
'{"title": "Test", "summary": "Summary", "confidence": 0.95}'
)
)
result = await llm.get_structured_response(
@@ -231,16 +402,36 @@ class TestNetworkErrorRetries:
)
assert result.title == "Test"
assert result.summary == "Summary"
assert call_count["count"] == 2
@pytest.mark.asyncio
async def test_network_error_exhausts_retries(self, test_settings):
"""Test that persistent network errors exhaust retry attempts"""
async def test_timeout_retry_exhausts_after_max_attempts(self, test_settings):
"""Test that timeout retry stops after max attempts"""
llm = LLM(settings=test_settings, temperature=0.4, max_tokens=100)
with patch("reflector.llm.Settings") as mock_settings:
mock_settings.llm.astructured_predict = AsyncMock(
side_effect=ConnectionError("Connection refused")
call_count = {"count": 0}
async def workflow_run_side_effect(*args, **kwargs):
call_count["count"] += 1
raise WorkflowTimeoutError("Operation timed out after 120 seconds")
with (
patch("reflector.llm.StructuredOutputWorkflow") as mock_workflow_class,
patch("reflector.llm.TreeSummarize") as mock_summarize,
patch("reflector.llm.Settings") as mock_settings,
):
mock_workflow = MagicMock()
mock_workflow.run = AsyncMock(side_effect=workflow_run_side_effect)
mock_workflow_class.return_value = mock_workflow
mock_summarizer = MagicMock()
mock_summarize.return_value = mock_summarizer
mock_summarizer.aget_response = AsyncMock(return_value="Some analysis")
mock_settings.llm.acomplete = AsyncMock(
return_value=make_completion_response(
'{"title": "Test", "summary": "Summary", "confidence": 0.95}'
)
)
with pytest.raises(RetryException, match="Retry attempts exceeded"):
@@ -248,129 +439,41 @@ class TestNetworkErrorRetries:
prompt="Test prompt", texts=["Test text"], output_cls=TestResponse
)
# 3 retry attempts
assert mock_settings.llm.astructured_predict.call_count == 3
class TestTextsInclusion:
"""Test that texts parameter is included in the prompt sent to astructured_predict"""
assert call_count["count"] == 3
@pytest.mark.asyncio
async def test_texts_included_in_prompt(self, test_settings):
"""Test that texts content is appended to the prompt for astructured_predict"""
async def test_timeout_retry_with_backoff(self, test_settings):
"""Test that exponential backoff is applied between retries"""
llm = LLM(settings=test_settings, temperature=0.4, max_tokens=100)
captured_prompts = []
call_times = []
async def capture_prompt(output_cls, prompt_tmpl, **kwargs):
captured_prompts.append(kwargs.get("user_prompt", ""))
return TestResponse(title="Test", summary="Summary", confidence=0.95)
with patch("reflector.llm.Settings") as mock_settings:
mock_settings.llm.astructured_predict = AsyncMock(
side_effect=capture_prompt
)
await llm.get_structured_response(
prompt="Identify all participants",
texts=["Alice: Hello everyone", "Bob: Hi Alice"],
output_cls=TestResponse,
)
assert len(captured_prompts) == 1
prompt_sent = captured_prompts[0]
assert "Identify all participants" in prompt_sent
assert "Alice: Hello everyone" in prompt_sent
assert "Bob: Hi Alice" in prompt_sent
@pytest.mark.asyncio
async def test_empty_texts_uses_prompt_only(self, test_settings):
"""Test that empty texts list sends only the prompt"""
llm = LLM(settings=test_settings, temperature=0.4, max_tokens=100)
captured_prompts = []
async def capture_prompt(output_cls, prompt_tmpl, **kwargs):
captured_prompts.append(kwargs.get("user_prompt", ""))
return TestResponse(title="Test", summary="Summary", confidence=0.95)
with patch("reflector.llm.Settings") as mock_settings:
mock_settings.llm.astructured_predict = AsyncMock(
side_effect=capture_prompt
)
await llm.get_structured_response(
prompt="Identify all participants",
texts=[],
output_cls=TestResponse,
)
assert len(captured_prompts) == 1
assert captured_prompts[0] == "Identify all participants"
@pytest.mark.asyncio
async def test_texts_included_in_reflection_retry(self, test_settings):
"""Test that texts are included in the prompt even during reflection retries"""
llm = LLM(settings=test_settings, temperature=0.4, max_tokens=100)
captured_prompts = []
call_count = {"count": 0}
async def capture_and_fail_first(output_cls, prompt_tmpl, **kwargs):
call_count["count"] += 1
captured_prompts.append(kwargs.get("user_prompt", ""))
if call_count["count"] == 1:
raise ValidationError.from_exception_data(
title="TestResponse",
line_errors=[
{
"type": "missing",
"loc": ("summary",),
"msg": "Field required",
"input": {},
}
],
async def workflow_run_side_effect(*args, **kwargs):
call_times.append(monotonic())
if len(call_times) < 3:
raise WorkflowTimeoutError("Operation timed out after 120 seconds")
return {
"success": TestResponse(
title="Test", summary="Summary", confidence=0.95
)
return TestResponse(title="Test", summary="Summary", confidence=0.95)
}
with patch("reflector.llm.Settings") as mock_settings:
mock_settings.llm.astructured_predict = AsyncMock(
side_effect=capture_and_fail_first
)
with (
patch("reflector.llm.StructuredOutputWorkflow") as mock_workflow_class,
patch("reflector.llm.TreeSummarize") as mock_summarize,
patch("reflector.llm.Settings") as mock_settings,
):
mock_workflow = MagicMock()
mock_workflow.run = AsyncMock(side_effect=workflow_run_side_effect)
mock_workflow_class.return_value = mock_workflow
await llm.get_structured_response(
prompt="Summarize this",
texts=["The meeting covered project updates"],
output_cls=TestResponse,
)
# Both first attempt and reflection retry should include the texts
assert len(captured_prompts) == 2
for prompt_sent in captured_prompts:
assert "Summarize this" in prompt_sent
assert "The meeting covered project updates" in prompt_sent
class TestReflectionRetryBackoff:
"""Test the reflection retry timing behavior"""
@pytest.mark.asyncio
async def test_value_error_triggers_reflection(self, test_settings):
"""Test that ValueError (parse failure) also triggers reflection retry"""
llm = LLM(settings=test_settings, temperature=0.4, max_tokens=100)
call_count = {"count": 0}
async def astructured_predict_handler(output_cls, prompt_tmpl, **kwargs):
call_count["count"] += 1
if call_count["count"] == 1:
raise ValueError("Could not parse output")
assert "reflection" in kwargs
return TestResponse(title="Test", summary="Summary", confidence=0.95)
with patch("reflector.llm.Settings") as mock_settings:
mock_settings.llm.astructured_predict = AsyncMock(
side_effect=astructured_predict_handler
mock_summarizer = MagicMock()
mock_summarize.return_value = mock_summarizer
mock_summarizer.aget_response = AsyncMock(return_value="Some analysis")
mock_settings.llm.acomplete = AsyncMock(
return_value=make_completion_response(
'{"title": "Test", "summary": "Summary", "confidence": 0.95}'
)
)
result = await llm.get_structured_response(
@@ -378,20 +481,8 @@ class TestReflectionRetryBackoff:
)
assert result.title == "Test"
assert call_count["count"] == 2
@pytest.mark.asyncio
async def test_format_validation_error_method(self, test_settings):
"""Test _format_validation_error produces correct feedback"""
# ValidationError
try:
TestResponse(title="x", summary="y", confidence=5.0) # confidence > 1
except ValidationError as e:
result = LLM._format_validation_error(e)
assert "Schema validation errors" in result
assert "confidence" in result
# ValueError
result = LLM._format_validation_error(ValueError("bad input"))
assert "Parse error:" in result
assert "bad input" in result
if len(call_times) >= 2:
time_between_calls = call_times[1] - call_times[0]
assert (
time_between_calls >= 1.5
), f"Expected ~2s backoff, got {time_between_calls}s"

View File

@@ -1,58 +0,0 @@
"""Tests for password hashing utilities."""
from reflector.auth.password_utils import hash_password, verify_password
def test_hash_and_verify():
pw = "my-secret-password"
h = hash_password(pw)
assert verify_password(pw, h) is True
def test_wrong_password():
h = hash_password("correct")
assert verify_password("wrong", h) is False
def test_hash_format():
h = hash_password("test")
parts = h.split("$")
assert len(parts) == 3
assert parts[0] == "pbkdf2:sha256:100000"
assert len(parts[1]) == 32 # 16 bytes hex = 32 chars
assert len(parts[2]) == 64 # sha256 hex = 64 chars
def test_different_salts():
h1 = hash_password("same")
h2 = hash_password("same")
assert h1 != h2 # different salts produce different hashes
assert verify_password("same", h1) is True
assert verify_password("same", h2) is True
def test_malformed_hash():
assert verify_password("test", "garbage") is False
assert verify_password("test", "") is False
assert verify_password("test", "pbkdf2:sha256:100000$short") is False
def test_empty_password():
h = hash_password("")
assert verify_password("", h) is True
assert verify_password("notempty", h) is False
def test_unicode_password():
pw = "p\u00e4ssw\u00f6rd\U0001f512"
h = hash_password(pw)
assert verify_password(pw, h) is True
assert verify_password("password", h) is False
def test_constant_time_comparison():
"""Verify that hmac.compare_digest is used (structural test)."""
import inspect
source = inspect.getsource(verify_password)
assert "hmac.compare_digest" in source

View File

@@ -27,5 +27,5 @@ WEBSOCKET_URL=ws://127.0.0.1:1250
AUTH_CALLBACK_URL=http://localhost:3000/auth-callback
# Sentry
# NEXT_PUBLIC_SENTRY_DSN=https://your-dsn@sentry.io/project-id
# SENTRY_DSN=https://your-dsn@sentry.io/project-id
# SENTRY_IGNORE_API_RESOLUTION_ERROR=1

View File

@@ -1,53 +0,0 @@
# =======================================================
# Reflector Self-Hosted Production — Frontend Configuration
# Generated by: ./scripts/setup-selfhosted.sh
# =======================================================
# Site URL — set to your domain or server IP
# The setup script auto-detects this on Linux.
SITE_URL=https://localhost
NEXTAUTH_URL=https://localhost
NEXTAUTH_SECRET=changeme-generate-a-secure-random-string
# API URLs
# Public-facing (what the browser uses):
API_URL=https://localhost
WEBSOCKET_URL=auto
# Internal Docker network (server-side rendering):
SERVER_API_URL=http://server:1250
KV_URL=redis://redis:6379
# Authentication
# Set to true when Authentik or password auth is configured
FEATURE_REQUIRE_LOGIN=false
# Auth provider: "authentik" or "credentials"
# Set to "credentials" when using password auth backend
# AUTH_PROVIDER=credentials
# Nullify auth vars when not using Authentik
AUTHENTIK_ISSUER=
AUTHENTIK_REFRESH_TOKEN_URL=
# =======================================================
# Authentik OAuth/OIDC (Optional)
# Uncomment and configure when enabling authentication.
# See docsv2/selfhosted-production.md for setup instructions.
# =======================================================
# FEATURE_REQUIRE_LOGIN=true
# AUTHENTIK_ISSUER=https://authentik.example.com/application/o/reflector
# AUTHENTIK_REFRESH_TOKEN_URL=https://authentik.example.com/application/o/token/
# AUTHENTIK_CLIENT_ID=your-client-id
# AUTHENTIK_CLIENT_SECRET=your-client-secret
# =======================================================
# Feature Flags
# =======================================================
# FEATURE_ROOMS=true
# FEATURE_BROWSE=true
# =======================================================
# Sentry (Optional)
# =======================================================
# NEXT_PUBLIC_SENTRY_DSN=

View File

@@ -78,10 +78,7 @@ const useMp3 = (transcriptId: string, waiting?: boolean): Mp3Response => {
// Audio is not deleted, proceed to load it
audioElement = document.createElement("audio");
const audioUrl = `${API_URL}/v1/transcripts/${transcriptId}/audio/mp3`;
audioElement.src = accessTokenInfo
? `${audioUrl}?token=${encodeURIComponent(accessTokenInfo)}`
: audioUrl;
audioElement.src = `${API_URL}/v1/transcripts/${transcriptId}/audio/mp3`;
audioElement.crossOrigin = "anonymous";
audioElement.preload = "auto";

View File

@@ -1,22 +1,18 @@
import { useEffect, useState } from "react";
import { Topic, FinalSummary, Status } from "./webSocketTypes";
import { useError } from "../../(errors)/errorContext";
import type { components, operations } from "../../reflector-api";
import type { components } from "../../reflector-api";
type AudioWaveform = components["schemas"]["AudioWaveform"];
type GetTranscriptSegmentTopic =
components["schemas"]["GetTranscriptSegmentTopic"];
import { useQueryClient } from "@tanstack/react-query";
import { WEBSOCKET_URL } from "../../lib/apiClient";
import { $api, WEBSOCKET_URL } from "../../lib/apiClient";
import {
invalidateTranscript,
invalidateTranscriptTopics,
invalidateTranscriptWaveform,
} from "../../lib/apiHooks";
import { useAuth } from "../../lib/AuthProvider";
import { parseNonEmptyString } from "../../lib/utils";
type TranscriptWsEvent =
operations["v1_transcript_get_websocket_events"]["responses"][200]["content"]["application/json"];
import { NonEmptyString } from "../../lib/utils";
export type UseWebSockets = {
transcriptTextLive: string;
@@ -31,7 +27,6 @@ export type UseWebSockets = {
};
export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
const auth = useAuth();
const [transcriptTextLive, setTranscriptTextLive] = useState<string>("");
const [translateText, setTranslateText] = useState<string>("");
const [title, setTitle] = useState<string>("");
@@ -336,168 +331,156 @@ export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
};
if (!transcriptId) return;
const tsId = parseNonEmptyString(transcriptId);
const MAX_RETRIES = 10;
const url = `${WEBSOCKET_URL}/v1/transcripts/${transcriptId}/events`;
let ws: WebSocket | null = null;
let retryCount = 0;
let retryTimeout: ReturnType<typeof setTimeout> | null = null;
let intentionalClose = false;
let ws = new WebSocket(url);
const connect = () => {
const subprotocols = auth.accessToken
? ["bearer", auth.accessToken]
: undefined;
ws = new WebSocket(url, subprotocols);
ws.onopen = () => {
console.debug("WebSocket connection opened");
};
ws.onopen = () => {
console.debug("WebSocket connection opened");
retryCount = 0;
};
ws.onmessage = (event) => {
const message = JSON.parse(event.data);
ws.onmessage = (event) => {
const message: TranscriptWsEvent = JSON.parse(event.data);
try {
switch (message.event) {
case "TRANSCRIPT":
const newText = (message.data.text ?? "").trim();
const newTranslation = (message.data.translation ?? "").trim();
try {
switch (message.event) {
case "TRANSCRIPT": {
const newText = (message.data.text ?? "").trim();
const newTranslation = (message.data.translation ?? "").trim();
if (!newText) break;
if (!newText) break;
console.debug("TRANSCRIPT event:", newText);
setTextQueue((prevQueue) => [...prevQueue, newText]);
setTranslationQueue((prevQueue) => [...prevQueue, newTranslation]);
console.debug("TRANSCRIPT event:", newText);
setTextQueue((prevQueue) => [...prevQueue, newText]);
setTranslationQueue((prevQueue) => [
...prevQueue,
newTranslation,
]);
setAccumulatedText((prevText) => prevText + " " + newText);
break;
setAccumulatedText((prevText) => prevText + " " + newText);
break;
}
case "TOPIC":
setTopics((prevTopics) => {
const topic = message.data;
const index = prevTopics.findIndex(
(prevTopic) => prevTopic.id === topic.id,
);
if (index >= 0) {
prevTopics[index] = topic;
return prevTopics;
}
setAccumulatedText((prevText) =>
prevText.slice(topic.transcript?.length ?? 0),
);
return [...prevTopics, topic];
});
console.debug("TOPIC event:", message.data);
invalidateTranscriptTopics(queryClient, tsId);
break;
case "FINAL_SHORT_SUMMARY":
console.debug("FINAL_SHORT_SUMMARY event:", message.data);
break;
case "FINAL_LONG_SUMMARY":
setFinalSummary({ summary: message.data.long_summary });
invalidateTranscript(queryClient, tsId);
break;
case "FINAL_TITLE":
console.debug("FINAL_TITLE event:", message.data);
setTitle(message.data.title);
invalidateTranscript(queryClient, tsId);
break;
case "WAVEFORM":
console.debug(
"WAVEFORM event length:",
message.data.waveform.length,
case "TOPIC":
setTopics((prevTopics) => {
const topic = message.data as Topic;
const index = prevTopics.findIndex(
(prevTopic) => prevTopic.id === topic.id,
);
setWaveForm({ data: message.data.waveform });
invalidateTranscriptWaveform(queryClient, tsId);
break;
case "DURATION":
console.debug("DURATION event:", message.data);
setDuration(message.data.duration);
break;
case "STATUS":
console.log("STATUS event:", message.data);
if (message.data.value === "error") {
setError(
Error("Websocket error status"),
"There was an error processing this meeting.",
);
if (index >= 0) {
prevTopics[index] = topic;
return prevTopics;
}
setStatus(message.data);
invalidateTranscript(queryClient, tsId);
if (message.data.value === "ended") {
intentionalClose = true;
ws?.close();
}
break;
case "ACTION_ITEMS":
console.debug("ACTION_ITEMS event:", message.data);
invalidateTranscript(queryClient, tsId);
break;
default: {
const _exhaustive: never = message;
console.warn(
`Received unknown WebSocket event: ${(_exhaustive as TranscriptWsEvent).event}`,
setAccumulatedText((prevText) =>
prevText.slice(topic.transcript.length),
);
}
}
} catch (error) {
setError(error);
}
};
ws.onerror = (error) => {
console.error("WebSocket error:", error);
};
ws.onclose = (event) => {
console.debug("WebSocket connection closed, code:", event.code);
if (intentionalClose) return;
const normalCodes = [1000, 1001, 1005];
if (normalCodes.includes(event.code)) return;
if (retryCount < MAX_RETRIES) {
const delay = Math.min(1000 * Math.pow(2, retryCount), 30000);
console.log(
`WebSocket reconnecting in ${delay}ms (attempt ${retryCount + 1}/${MAX_RETRIES})`,
);
if (retryCount === 0) {
setError(
new Error("WebSocket connection lost"),
"Connection lost. Reconnecting...",
return [...prevTopics, topic];
});
console.debug("TOPIC event:", message.data);
// Invalidate topics query to sync with WebSocket data
invalidateTranscriptTopics(
queryClient,
transcriptId as NonEmptyString,
);
}
retryCount++;
retryTimeout = setTimeout(connect, delay);
} else {
break;
case "FINAL_SHORT_SUMMARY":
console.debug("FINAL_SHORT_SUMMARY event:", message.data);
break;
case "FINAL_LONG_SUMMARY":
if (message.data) {
setFinalSummary(message.data);
// Invalidate transcript query to sync summary
invalidateTranscript(queryClient, transcriptId as NonEmptyString);
}
break;
case "FINAL_TITLE":
console.debug("FINAL_TITLE event:", message.data);
if (message.data) {
setTitle(message.data.title);
// Invalidate transcript query to sync title
invalidateTranscript(queryClient, transcriptId as NonEmptyString);
}
break;
case "WAVEFORM":
console.debug(
"WAVEFORM event length:",
message.data.waveform.length,
);
if (message.data) {
setWaveForm(message.data.waveform);
invalidateTranscriptWaveform(
queryClient,
transcriptId as NonEmptyString,
);
}
break;
case "DURATION":
console.debug("DURATION event:", message.data);
if (message.data) {
setDuration(message.data.duration);
}
break;
case "STATUS":
console.log("STATUS event:", message.data);
if (message.data.value === "error") {
setError(
Error("Websocket error status"),
"There was an error processing this meeting.",
);
}
setStatus(message.data);
invalidateTranscript(queryClient, transcriptId as NonEmptyString);
if (message.data.value === "ended") {
ws.close();
}
break;
default:
setError(
new Error(`Received unknown WebSocket event: ${message.event}`),
);
}
} catch (error) {
setError(error);
}
};
ws.onerror = (error) => {
console.error("WebSocket error:", error);
setError(new Error("A WebSocket error occurred."));
};
ws.onclose = (event) => {
console.debug("WebSocket connection closed");
switch (event.code) {
case 1000: // Normal Closure:
break;
case 1005: // Closure by client FF
break;
case 1001: // Navigate away
break;
case 1006: // Closed by client Chrome
console.warn(
"WebSocket closed by client, likely duplicated connection in react dev mode",
);
break;
default:
setError(
new Error(`WebSocket closed unexpectedly with code: ${event.code}`),
"Disconnected from the server. Please refresh the page.",
);
}
};
console.log(
"Socket is closed. Reconnect will be attempted in 1 second.",
event.reason,
);
// todo handle reconnect with socket.io
}
};
connect();
return () => {
intentionalClose = true;
if (retryTimeout) clearTimeout(retryTimeout);
ws?.close();
ws.close();
};
}, [transcriptId]);

View File

@@ -23,7 +23,7 @@ export default function UserInfo() {
className="font-light px-2"
onClick={(e) => {
e.preventDefault();
auth.signIn();
auth.signIn("authentik");
}}
>
Log in

View File

@@ -4,12 +4,14 @@ import React, { useEffect, useRef } from "react";
import { useQueryClient } from "@tanstack/react-query";
import { WEBSOCKET_URL } from "./apiClient";
import { useAuth } from "./AuthProvider";
import { invalidateTranscript, invalidateTranscriptLists } from "./apiHooks";
import { parseNonEmptyString } from "./utils";
import type { operations } from "../reflector-api";
import { z } from "zod";
import { invalidateTranscriptLists, TRANSCRIPT_SEARCH_URL } from "./apiHooks";
type UserWsEvent =
operations["v1_user_get_websocket_events"]["responses"][200]["content"]["application/json"];
const UserEvent = z.object({
event: z.string(),
});
type UserEvent = z.TypeOf<typeof UserEvent>;
class UserEventsStore {
private socket: WebSocket | null = null;
@@ -131,26 +133,23 @@ export function UserEventsProvider({
if (!detachRef.current) {
const onMessage = (event: MessageEvent) => {
try {
const msg: UserWsEvent = JSON.parse(event.data);
const msg = UserEvent.parse(JSON.parse(event.data));
const eventName = msg.event;
switch (msg.event) {
const invalidateList = () => invalidateTranscriptLists(queryClient);
switch (eventName) {
case "TRANSCRIPT_CREATED":
case "TRANSCRIPT_DELETED":
case "TRANSCRIPT_STATUS":
case "TRANSCRIPT_FINAL_TITLE":
case "TRANSCRIPT_DURATION":
invalidateTranscriptLists(queryClient).then(() => {});
invalidateTranscript(
queryClient,
parseNonEmptyString(msg.data.id),
).then(() => {});
invalidateList().then(() => {});
break;
default:
// Ignore other content events for list updates
break;
default: {
const _exhaustive: never = msg;
console.warn(
`Unknown user event: ${(_exhaustive as UserWsEvent).event}`,
);
}
}
} catch (err) {
console.warn("Invalid user event message", event.data);

View File

@@ -13,33 +13,9 @@ export const API_URL = !isBuildPhase
? getClientEnv().API_URL
: "http://localhost";
/**
* Derive a WebSocket URL from the API_URL.
* Handles full URLs (http://host/api, https://host/api) and relative paths (/api).
* For full URLs, ws/wss is derived from the URL's own protocol.
* For relative URLs, ws/wss is derived from window.location.protocol.
*/
const deriveWebSocketUrl = (apiUrl: string): string => {
if (typeof window === "undefined") {
return "ws://localhost";
}
const parsed = new URL(apiUrl, window.location.origin);
const wsProtocol = parsed.protocol === "https:" ? "wss:" : "ws:";
// Normalize: remove trailing slash from pathname
const pathname = parsed.pathname.replace(/\/+$/, "");
return `${wsProtocol}//${parsed.host}${pathname}`;
};
const resolveWebSocketUrl = (): string => {
if (isBuildPhase) return "ws://localhost";
const raw = getClientEnv().WEBSOCKET_URL;
if (!raw || raw === "auto") {
return deriveWebSocketUrl(API_URL);
}
return raw;
};
export const WEBSOCKET_URL = resolveWebSocketUrl();
export const WEBSOCKET_URL = !isBuildPhase
? getClientEnv().WEBSOCKET_URL || "ws://127.0.0.1:1250"
: "ws://localhost";
export const client = createClient<paths>({
baseUrl: API_URL,

View File

@@ -7,7 +7,6 @@ import type { components } from "../reflector-api";
import { useAuth } from "./AuthProvider";
import { MeetingId } from "./types";
import { NonEmptyString } from "./utils";
import type { TranscriptStatus } from "./transcript";
/*
* XXX error types returned from the hooks are not always correct; declared types are ValidationError but real type could be string or any other
@@ -105,12 +104,6 @@ export function useTranscriptProcess() {
});
}
const ACTIVE_TRANSCRIPT_STATUSES = new Set<TranscriptStatus>([
"processing",
"uploaded",
"recording",
]);
export function useTranscriptGet(transcriptId: NonEmptyString | null) {
return $api.useQuery(
"get",
@@ -124,10 +117,6 @@ export function useTranscriptGet(transcriptId: NonEmptyString | null) {
},
{
enabled: !!transcriptId,
refetchInterval: (query) => {
const status = query.state.data?.status;
return status && ACTIVE_TRANSCRIPT_STATUSES.has(status) ? 5000 : false;
},
},
);
}

View File

@@ -1,6 +1,5 @@
import { AuthOptions } from "next-auth";
import AuthentikProvider from "next-auth/providers/authentik";
import CredentialsProvider from "next-auth/providers/credentials";
import type { JWT } from "next-auth/jwt";
import { JWTWithAccessToken, CustomSession } from "./types";
import {
@@ -53,7 +52,7 @@ const TOKEN_CACHE_TTL = REFRESH_ACCESS_TOKEN_BEFORE;
const getAuthentikClientId = () => getNextEnvVar("AUTHENTIK_CLIENT_ID");
const getAuthentikClientSecret = () => getNextEnvVar("AUTHENTIK_CLIENT_SECRET");
const getAuthentikRefreshTokenUrl = () =>
getNextEnvVar("AUTHENTIK_REFRESH_TOKEN_URL").replace(/\/+$/, "");
getNextEnvVar("AUTHENTIK_REFRESH_TOKEN_URL");
const getAuthentikIssuer = () => {
const stringUrl = getNextEnvVar("AUTHENTIK_ISSUER");
@@ -62,194 +61,113 @@ const getAuthentikIssuer = () => {
} catch (e) {
throw new Error("AUTHENTIK_ISSUER is not a valid URL: " + stringUrl);
}
return stringUrl.replace(/\/+$/, "");
return stringUrl;
};
export const authOptions = (): AuthOptions => {
if (!featureEnabled("requireLogin")) {
return { providers: [] };
}
const authProvider = process.env.AUTH_PROVIDER;
if (authProvider === "credentials") {
return credentialsAuthOptions();
}
return authentikAuthOptions();
};
function credentialsAuthOptions(): AuthOptions {
return {
providers: [
CredentialsProvider({
name: "Password",
credentials: {
email: { label: "Email", type: "email" },
password: { label: "Password", type: "password" },
},
async authorize(credentials) {
if (!credentials?.email || !credentials?.password) return null;
const apiUrl = getNextEnvVar("SERVER_API_URL");
const response = await fetch(`${apiUrl}/v1/auth/login`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
email: credentials.email,
password: credentials.password,
}),
});
if (!response.ok) return null;
const data = await response.json();
return {
id: "pending",
email: credentials.email,
accessToken: data.access_token,
expiresIn: data.expires_in,
};
},
}),
],
session: { strategy: "jwt" },
pages: {
signIn: "/login",
},
callbacks: {
async jwt({ token, user }) {
if (user) {
// First login - user comes from authorize()
const typedUser = user as any;
token.accessToken = typedUser.accessToken;
token.accessTokenExpires = Date.now() + typedUser.expiresIn * 1000;
// Resolve actual user ID from backend
const userId = await getUserId(typedUser.accessToken);
if (userId) {
token.sub = userId;
}
token.email = typedUser.email;
}
return token;
},
async session({ session, token }) {
const extendedToken = token as JWTWithAccessToken;
return {
...session,
accessToken: extendedToken.accessToken,
accessTokenExpires: extendedToken.accessTokenExpires,
error: extendedToken.error,
user: {
id: assertExistsAndNonEmptyString(token.sub, "User ID required"),
name: extendedToken.name,
email: extendedToken.email,
},
} satisfies CustomSession;
},
},
};
}
function authentikAuthOptions(): AuthOptions {
return {
providers: [
AuthentikProvider({
...(() => {
const [clientId, clientSecret, issuer] = sequenceThrows(
getAuthentikClientId,
getAuthentikClientSecret,
getAuthentikIssuer,
);
return {
clientId,
clientSecret,
issuer,
};
})(),
authorization: {
params: {
scope: "openid email profile offline_access",
},
},
}),
],
session: {
strategy: "jwt",
},
callbacks: {
async jwt({ token, account, user }) {
if (account && !account.access_token) {
await deleteTokenCache(tokenCacheRedis, `token:${token.sub}`);
}
if (account && user) {
// called only on first login
// XXX account.expires_in used in example is not defined for authentik backend, but expires_at is
if (account.access_token) {
const expiresAtS = assertExists(account.expires_at);
const expiresAtMs = expiresAtS * 1000;
const jwtToken: JWTWithAccessToken = {
...token,
accessToken: account.access_token,
accessTokenExpires: expiresAtMs,
refreshToken: account.refresh_token,
};
if (jwtToken.error) {
await deleteTokenCache(tokenCacheRedis, `token:${token.sub}`);
} else {
assertNotExists(
jwtToken.error,
`panic! trying to cache token with error in jwt: ${jwtToken.error}`,
export const authOptions = (): AuthOptions =>
featureEnabled("requireLogin")
? {
providers: [
AuthentikProvider({
...(() => {
const [clientId, clientSecret, issuer] = sequenceThrows(
getAuthentikClientId,
getAuthentikClientSecret,
getAuthentikIssuer,
);
await setTokenCache(tokenCacheRedis, `token:${token.sub}`, {
token: jwtToken,
timestamp: Date.now(),
});
return jwtToken;
return {
clientId,
clientSecret,
issuer,
};
})(),
authorization: {
params: {
scope: "openid email profile offline_access",
},
},
}),
],
session: {
strategy: "jwt",
},
callbacks: {
async jwt({ token, account, user }) {
if (account && !account.access_token) {
await deleteTokenCache(tokenCacheRedis, `token:${token.sub}`);
}
}
}
const currentToken = await getTokenCache(
tokenCacheRedis,
`token:${token.sub}`,
);
console.debug(
"currentToken from cache",
JSON.stringify(currentToken, null, 2),
"will be returned?",
currentToken &&
!shouldRefreshToken(currentToken.token.accessTokenExpires),
);
if (
currentToken &&
!shouldRefreshToken(currentToken.token.accessTokenExpires)
) {
return currentToken.token;
}
if (account && user) {
// called only on first login
// XXX account.expires_in used in example is not defined for authentik backend, but expires_at is
if (account.access_token) {
const expiresAtS = assertExists(account.expires_at);
const expiresAtMs = expiresAtS * 1000;
const jwtToken: JWTWithAccessToken = {
...token,
accessToken: account.access_token,
accessTokenExpires: expiresAtMs,
refreshToken: account.refresh_token,
};
if (jwtToken.error) {
await deleteTokenCache(tokenCacheRedis, `token:${token.sub}`);
} else {
assertNotExists(
jwtToken.error,
`panic! trying to cache token with error in jwt: ${jwtToken.error}`,
);
await setTokenCache(tokenCacheRedis, `token:${token.sub}`, {
token: jwtToken,
timestamp: Date.now(),
});
return jwtToken;
}
}
}
// access token has expired, try to update it
return await lockedRefreshAccessToken(token);
},
async session({ session, token }) {
const extendedToken = token as JWTWithAccessToken;
console.log("extendedToken", extendedToken);
const userId = await getUserId(extendedToken.accessToken);
const currentToken = await getTokenCache(
tokenCacheRedis,
`token:${token.sub}`,
);
console.debug(
"currentToken from cache",
JSON.stringify(currentToken, null, 2),
"will be returned?",
currentToken &&
!shouldRefreshToken(currentToken.token.accessTokenExpires),
);
if (
currentToken &&
!shouldRefreshToken(currentToken.token.accessTokenExpires)
) {
return currentToken.token;
}
return {
...session,
accessToken: extendedToken.accessToken,
accessTokenExpires: extendedToken.accessTokenExpires,
error: extendedToken.error,
user: {
id: assertExistsAndNonEmptyString(userId, "User ID required"),
name: extendedToken.name,
email: extendedToken.email,
// access token has expired, try to update it
return await lockedRefreshAccessToken(token);
},
} satisfies CustomSession;
},
},
};
}
async session({ session, token }) {
const extendedToken = token as JWTWithAccessToken;
console.log("extendedToken", extendedToken);
const userId = await getUserId(extendedToken.accessToken);
return {
...session,
accessToken: extendedToken.accessToken,
accessTokenExpires: extendedToken.accessTokenExpires,
error: extendedToken.error,
user: {
id: assertExistsAndNonEmptyString(userId, "User ID required"),
name: extendedToken.name,
email: extendedToken.email,
},
} satisfies CustomSession;
},
},
}
: {
providers: [],
};
async function lockedRefreshAccessToken(
token: JWT,

View File

@@ -2,7 +2,6 @@ import {
assertExists,
assertExistsAndNonEmptyString,
NonEmptyString,
parseMaybeNonEmptyString,
parseNonEmptyString,
} from "./utils";
import { isBuildPhase } from "./next";
@@ -28,13 +27,10 @@ export type EnvFeaturePartial = {
[key in FeatureEnvName]: boolean | null;
};
export type AuthProviderType = "authentik" | "credentials" | null;
// CONTRACT: isomorphic with JSON.stringify
export type ClientEnvCommon = EnvFeaturePartial & {
API_URL: NonEmptyString;
WEBSOCKET_URL: NonEmptyString | null;
AUTH_PROVIDER: AuthProviderType;
};
let clientEnv: ClientEnvCommon | null = null;
@@ -62,12 +58,6 @@ const parseBooleanString = (str: string | undefined): boolean | null => {
return str === "true";
};
const parseAuthProvider = (): AuthProviderType => {
const val = process.env.AUTH_PROVIDER;
if (val === "authentik" || val === "credentials") return val;
return null;
};
export const getClientEnvServer = (): ClientEnvCommon => {
if (typeof window !== "undefined") {
throw new Error(
@@ -84,16 +74,14 @@ export const getClientEnvServer = (): ClientEnvCommon => {
if (isBuildPhase) {
return {
API_URL: getNextEnvVar("API_URL"),
WEBSOCKET_URL: parseMaybeNonEmptyString(process.env.WEBSOCKET_URL ?? ""),
AUTH_PROVIDER: parseAuthProvider(),
WEBSOCKET_URL: getNextEnvVar("WEBSOCKET_URL"),
...features,
};
}
clientEnv = {
API_URL: getNextEnvVar("API_URL"),
WEBSOCKET_URL: parseMaybeNonEmptyString(process.env.WEBSOCKET_URL ?? ""),
AUTH_PROVIDER: parseAuthProvider(),
WEBSOCKET_URL: getNextEnvVar("WEBSOCKET_URL"),
...features,
};
return clientEnv;

View File

@@ -1,76 +0,0 @@
"use client";
import { useState } from "react";
import { signIn } from "next-auth/react";
import { useRouter } from "next/navigation";
import {
Box,
Button,
Field,
Input,
VStack,
Text,
Heading,
} from "@chakra-ui/react";
export default function LoginPage() {
const router = useRouter();
const [email, setEmail] = useState("");
const [password, setPassword] = useState("");
const [error, setError] = useState<string | null>(null);
const [loading, setLoading] = useState(false);
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
setError(null);
setLoading(true);
const result = await signIn("credentials", {
email,
password,
redirect: false,
});
setLoading(false);
if (result?.error) {
console.log(result?.error);
setError("Invalid email or password");
} else {
router.push("/");
}
};
return (
<Box maxW="400px" mx="auto" mt="100px" p={6}>
<VStack gap={6} as="form" onSubmit={handleSubmit}>
<Heading size="lg">Log in</Heading>
{error && <Text color="red.500">{error}</Text>}
<Field.Root required>
<Field.Label>Email</Field.Label>
<Input
type="email"
value={email}
onChange={(e) => setEmail(e.target.value)}
/>
</Field.Root>
<Field.Root required>
<Field.Label>Password</Field.Label>
<Input
type="password"
value={password}
onChange={(e) => setPassword(e.target.value)}
/>
</Field.Root>
<Button
type="submit"
colorPalette="blue"
width="full"
loading={loading}
>
Log in
</Button>
</VStack>
</Box>
);
}

View File

@@ -568,10 +568,7 @@ export interface paths {
path?: never;
cookie?: never;
};
/**
* Transcript WebSocket event schema
* @description Stub exposing the discriminated union of all transcript-level WS events for OpenAPI type generation. Real events are delivered over the WebSocket at the same path.
*/
/** Transcript Get Websocket Events */
get: operations["v1_transcript_get_websocket_events"];
put?: never;
post?: never;
@@ -667,26 +664,6 @@ export interface paths {
patch?: never;
trace?: never;
};
"/v1/events": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
/**
* User WebSocket event schema
* @description Stub exposing the discriminated union of all user-level WS events for OpenAPI type generation. Real events are delivered over the WebSocket at the same path.
*/
get: operations["v1_user_get_websocket_events"];
put?: never;
post?: never;
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/v1/zulip/streams": {
parameters: {
query?: never;
@@ -1032,8 +1009,6 @@ export interface components {
room_name?: string | null;
/** Audio Deleted */
audio_deleted?: boolean | null;
/** Change Seq */
change_seq?: number | null;
};
/** GetTranscriptSegmentTopic */
GetTranscriptSegmentTopic: {
@@ -1180,8 +1155,6 @@ export interface components {
room_name?: string | null;
/** Audio Deleted */
audio_deleted?: boolean | null;
/** Change Seq */
change_seq?: number | null;
/** Participants */
participants:
| components["schemas"]["TranscriptParticipantWithEmail"][]
@@ -1245,8 +1218,6 @@ export interface components {
room_name?: string | null;
/** Audio Deleted */
audio_deleted?: boolean | null;
/** Change Seq */
change_seq?: number | null;
/** Participants */
participants:
| components["schemas"]["TranscriptParticipantWithEmail"][]
@@ -1311,8 +1282,6 @@ export interface components {
room_name?: string | null;
/** Audio Deleted */
audio_deleted?: boolean | null;
/** Change Seq */
change_seq?: number | null;
/** Participants */
participants:
| components["schemas"]["TranscriptParticipantWithEmail"][]
@@ -1384,8 +1353,6 @@ export interface components {
room_name?: string | null;
/** Audio Deleted */
audio_deleted?: boolean | null;
/** Change Seq */
change_seq?: number | null;
/** Participants */
participants:
| components["schemas"]["TranscriptParticipantWithEmail"][]
@@ -1459,8 +1426,6 @@ export interface components {
room_name?: string | null;
/** Audio Deleted */
audio_deleted?: boolean | null;
/** Change Seq */
change_seq?: number | null;
/** Participants */
participants:
| components["schemas"]["TranscriptParticipantWithEmail"][]
@@ -1846,8 +1811,6 @@ export interface components {
* @default 0
*/
total_match_count: number;
/** Change Seq */
change_seq?: number | null;
};
/**
* SourceKind
@@ -1914,33 +1877,6 @@ export interface components {
/** Name */
name: string;
};
/** TranscriptActionItems */
TranscriptActionItems: {
/** Action Items */
action_items: {
[key: string]: unknown;
};
};
/** TranscriptDuration */
TranscriptDuration: {
/** Duration */
duration: number;
};
/** TranscriptFinalLongSummary */
TranscriptFinalLongSummary: {
/** Long Summary */
long_summary: string;
};
/** TranscriptFinalShortSummary */
TranscriptFinalShortSummary: {
/** Short Summary */
short_summary: string;
};
/** TranscriptFinalTitle */
TranscriptFinalTitle: {
/** Title */
title: string;
};
/** TranscriptParticipant */
TranscriptParticipant: {
/** Id */
@@ -1981,113 +1917,6 @@ export interface components {
/** End */
end: number;
};
/** TranscriptText */
TranscriptText: {
/** Text */
text: string;
/** Translation */
translation: string | null;
};
/** TranscriptWaveform */
TranscriptWaveform: {
/** Waveform */
waveform: number[];
};
/** TranscriptWsActionItems */
TranscriptWsActionItems: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "ACTION_ITEMS";
data: components["schemas"]["TranscriptActionItems"];
};
/** TranscriptWsDuration */
TranscriptWsDuration: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "DURATION";
data: components["schemas"]["TranscriptDuration"];
};
/** TranscriptWsFinalLongSummary */
TranscriptWsFinalLongSummary: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "FINAL_LONG_SUMMARY";
data: components["schemas"]["TranscriptFinalLongSummary"];
};
/** TranscriptWsFinalShortSummary */
TranscriptWsFinalShortSummary: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "FINAL_SHORT_SUMMARY";
data: components["schemas"]["TranscriptFinalShortSummary"];
};
/** TranscriptWsFinalTitle */
TranscriptWsFinalTitle: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "FINAL_TITLE";
data: components["schemas"]["TranscriptFinalTitle"];
};
/** TranscriptWsStatus */
TranscriptWsStatus: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "STATUS";
data: components["schemas"]["TranscriptWsStatusData"];
};
/** TranscriptWsStatusData */
TranscriptWsStatusData: {
/**
* Value
* @enum {string}
*/
value:
| "idle"
| "uploaded"
| "recording"
| "processing"
| "error"
| "ended";
};
/** TranscriptWsTopic */
TranscriptWsTopic: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "TOPIC";
data: components["schemas"]["GetTranscriptTopic"];
};
/** TranscriptWsTranscript */
TranscriptWsTranscript: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "TRANSCRIPT";
data: components["schemas"]["TranscriptText"];
};
/** TranscriptWsWaveform */
TranscriptWsWaveform: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "WAVEFORM";
data: components["schemas"]["TranscriptWaveform"];
};
/** UpdateParticipant */
UpdateParticipant: {
/** Speaker */
@@ -2158,109 +1987,6 @@ export interface components {
/** Email */
email: string | null;
};
/** UserTranscriptCreatedData */
UserTranscriptCreatedData: {
/**
* Id
* @description A non-empty string
*/
id: string;
};
/** UserTranscriptDeletedData */
UserTranscriptDeletedData: {
/**
* Id
* @description A non-empty string
*/
id: string;
};
/** UserTranscriptDurationData */
UserTranscriptDurationData: {
/**
* Id
* @description A non-empty string
*/
id: string;
/** Duration */
duration: number;
};
/** UserTranscriptFinalTitleData */
UserTranscriptFinalTitleData: {
/**
* Id
* @description A non-empty string
*/
id: string;
/**
* Title
* @description A non-empty string
*/
title: string;
};
/** UserTranscriptStatusData */
UserTranscriptStatusData: {
/**
* Id
* @description A non-empty string
*/
id: string;
/**
* Value
* @enum {string}
*/
value:
| "idle"
| "uploaded"
| "recording"
| "processing"
| "error"
| "ended";
};
/** UserWsTranscriptCreated */
UserWsTranscriptCreated: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "TRANSCRIPT_CREATED";
data: components["schemas"]["UserTranscriptCreatedData"];
};
/** UserWsTranscriptDeleted */
UserWsTranscriptDeleted: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "TRANSCRIPT_DELETED";
data: components["schemas"]["UserTranscriptDeletedData"];
};
/** UserWsTranscriptDuration */
UserWsTranscriptDuration: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "TRANSCRIPT_DURATION";
data: components["schemas"]["UserTranscriptDurationData"];
};
/** UserWsTranscriptFinalTitle */
UserWsTranscriptFinalTitle: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "TRANSCRIPT_FINAL_TITLE";
data: components["schemas"]["UserTranscriptFinalTitleData"];
};
/** UserWsTranscriptStatus */
UserWsTranscriptStatus: {
/**
* @description discriminator enum property added by openapi-typescript
* @enum {string}
*/
event: "TRANSCRIPT_STATUS";
data: components["schemas"]["UserTranscriptStatusData"];
};
/** ValidationError */
ValidationError: {
/** Location */
@@ -2967,8 +2693,6 @@ export interface operations {
source_kind?: components["schemas"]["SourceKind"] | null;
room_id?: string | null;
search_term?: string | null;
change_seq_from?: number | null;
sort_by?: ("created_at" | "change_seq") | null;
/** @description Page number */
page?: number;
/** @description Page size */
@@ -3699,16 +3423,7 @@ export interface operations {
[name: string]: unknown;
};
content: {
"application/json":
| components["schemas"]["TranscriptWsTranscript"]
| components["schemas"]["TranscriptWsTopic"]
| components["schemas"]["TranscriptWsStatus"]
| components["schemas"]["TranscriptWsFinalTitle"]
| components["schemas"]["TranscriptWsFinalLongSummary"]
| components["schemas"]["TranscriptWsFinalShortSummary"]
| components["schemas"]["TranscriptWsActionItems"]
| components["schemas"]["TranscriptWsDuration"]
| components["schemas"]["TranscriptWsWaveform"];
"application/json": unknown;
};
};
/** @description Validation Error */
@@ -3892,31 +3607,6 @@ export interface operations {
};
};
};
v1_user_get_websocket_events: {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json":
| components["schemas"]["UserWsTranscriptCreated"]
| components["schemas"]["UserWsTranscriptDeleted"]
| components["schemas"]["UserWsTranscriptStatus"]
| components["schemas"]["UserWsTranscriptFinalTitle"]
| components["schemas"]["UserWsTranscriptDuration"];
};
};
};
};
v1_zulip_get_streams: {
parameters: {
query?: never;

View File

@@ -4,7 +4,7 @@
import * as Sentry from "@sentry/nextjs";
const SENTRY_DSN = process.env.NEXT_PUBLIC_SENTRY_DSN;
const SENTRY_DSN = process.env.SENTRY_DSN;
if (SENTRY_DSN) {
Sentry.init({

View File

@@ -5,7 +5,7 @@
import * as Sentry from "@sentry/nextjs";
const SENTRY_DSN = process.env.NEXT_PUBLIC_SENTRY_DSN;
const SENTRY_DSN = process.env.SENTRY_DSN;
if (SENTRY_DSN) {
Sentry.init({

View File

@@ -4,7 +4,7 @@
import * as Sentry from "@sentry/nextjs";
const SENTRY_DSN = process.env.NEXT_PUBLIC_SENTRY_DSN;
const SENTRY_DSN = process.env.SENTRY_DSN;
if (SENTRY_DSN) {
Sentry.init({

View File

@@ -1,4 +0,0 @@
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1