selfhostyourtech/reflector
2
0
Fork 0
mirror of https://github.com/Monadical-SAS/reflector.git synced 2025-12-20 12:19:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge main into jisti-integration branch

Browse Source 
- Resolved conflicts in server/reflector/views/rooms.py to keep platform-agnostic approach
- Resolved conflicts in www/app/[roomName]/page.tsx to keep VideoPlatformEmbed approach
- Accepted main's version of generated API files (schemas.gen.ts, services.gen.ts, types.gen.ts)
- Removed config-template.ts as per main branch changes
This commit is contained in:
Mathieu Virbel
2025-09-15 12:53:49 -06:00
parent 293f7d4f1f c546e69739
commit 95e8011975
127 changed files with 11011 additions and 8821 deletions
Download Patch File Download Diff File Expand all files Collapse all files

45
.github/workflows/test_next_server.yml vendored Normal file
View File

@@ -0,0 +1,45 @@
name: Test Next Server
on:
pull_request:
paths:
- "www/**"
push:
branches:
- main
paths:
- "www/**"
jobs:
test-next-server:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./www
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install pnpm
uses: pnpm/action-setup@v4
with:
version: 8
- name: Setup Node.js cache
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'pnpm'
cache-dependency-path: './www/pnpm-lock.yaml'
- name: Install dependencies
run: pnpm install
- name: Run tests
run: pnpm test

1
.gitignore vendored
View File

@@ -17,3 +17,4 @@ server/test.sqlite
CLAUDE.local.md
www/.env.development
www/.env.production
.playwright-mcp

32
CHANGELOG.md
View File

@@ -1,5 +1,37 @@
# Changelog
## [0.10.0](https://github.com/Monadical-SAS/reflector/compare/v0.9.0...v0.10.0) (2025-09-11)
### Features
* replace nextjs-config with environment variables ([#632](https://github.com/Monadical-SAS/reflector/issues/632)) ([369ecdf](https://github.com/Monadical-SAS/reflector/commit/369ecdff13f3862d926a9c0b87df52c9d94c4dde))
### Bug Fixes
* anonymous users transcript permissions ([#621](https://github.com/Monadical-SAS/reflector/issues/621)) ([f81fe99](https://github.com/Monadical-SAS/reflector/commit/f81fe9948a9237b3e0001b2d8ca84f54d76878f9))
* auth post ([#624](https://github.com/Monadical-SAS/reflector/issues/624)) ([cde99ca](https://github.com/Monadical-SAS/reflector/commit/cde99ca2716f84ba26798f289047732f0448742e))
* auth post ([#626](https://github.com/Monadical-SAS/reflector/issues/626)) ([3b85ff3](https://github.com/Monadical-SAS/reflector/commit/3b85ff3bdf4fb053b103070646811bc990c0e70a))
* auth post ([#627](https://github.com/Monadical-SAS/reflector/issues/627)) ([962038e](https://github.com/Monadical-SAS/reflector/commit/962038ee3f2a555dc3c03856be0e4409456e0996))
* missing follow_redirects=True on modal endpoint ([#630](https://github.com/Monadical-SAS/reflector/issues/630)) ([fc363bd](https://github.com/Monadical-SAS/reflector/commit/fc363bd49b17b075e64f9186e5e0185abc325ea7))
* sync backend and frontend token refresh logic ([#614](https://github.com/Monadical-SAS/reflector/issues/614)) ([5a5b323](https://github.com/Monadical-SAS/reflector/commit/5a5b3233820df9536da75e87ce6184a983d4713a))
## [0.9.0](https://github.com/Monadical-SAS/reflector/compare/v0.8.2...v0.9.0) (2025-09-06)
### Features
* frontend openapi react query ([#606](https://github.com/Monadical-SAS/reflector/issues/606)) ([c4d2825](https://github.com/Monadical-SAS/reflector/commit/c4d2825c81f81ad8835629fbf6ea8c7383f8c31b))
### Bug Fixes
* align whisper transcriber api with parakeet ([#602](https://github.com/Monadical-SAS/reflector/issues/602)) ([0663700](https://github.com/Monadical-SAS/reflector/commit/0663700a615a4af69a03c96c410f049e23ec9443))
* kv use tls explicit ([#610](https://github.com/Monadical-SAS/reflector/issues/610)) ([08d88ec](https://github.com/Monadical-SAS/reflector/commit/08d88ec349f38b0d13e0fa4cb73486c8dfd31836))
* source kind for file processing ([#601](https://github.com/Monadical-SAS/reflector/issues/601)) ([dc82f8b](https://github.com/Monadical-SAS/reflector/commit/dc82f8bb3bdf3ab3d4088e592a30fd63907319e1))
* token refresh locking ([#613](https://github.com/Monadical-SAS/reflector/issues/613)) ([7f5a4c9](https://github.com/Monadical-SAS/reflector/commit/7f5a4c9ddc7fd098860c8bdda2ca3b57f63ded2f))
## [0.8.2](https://github.com/Monadical-SAS/reflector/compare/v0.8.1...v0.8.2) (2025-08-29)

1
CLAUDE.md
View File

@@ -66,7 +66,6 @@ pnpm install
# Copy configuration templates
cp .env_template .env
cp config-template.ts config.ts
```
**Development:**

36
README.md
View File

@@ -99,11 +99,10 @@ Start with `cd www`.
```bash
pnpm install
cp .env_template .env
cp config-template.ts config.ts
cp .env.example .env
```
Then, fill in the environment variables in `.env` and the configuration in `config.ts` as needed. If you are unsure on how to proceed, ask in Zulip.
Then, fill in the environment variables in `.env` as needed. If you are unsure on how to proceed, ask in Zulip.
**Run in development mode**
@@ -168,3 +167,34 @@ You can manually process an audio file by calling the process tool:
```bash
uv run python -m reflector.tools.process path/to/audio.wav
```
## Feature Flags
Reflector uses environment variable-based feature flags to control application functionality. These flags allow you to enable or disable features without code changes.
### Available Feature Flags
| Feature Flag | Environment Variable |
|-------------|---------------------|
| `requireLogin` | `NEXT_PUBLIC_FEATURE_REQUIRE_LOGIN` |
| `privacy` | `NEXT_PUBLIC_FEATURE_PRIVACY` |
| `browse` | `NEXT_PUBLIC_FEATURE_BROWSE` |
| `sendToZulip` | `NEXT_PUBLIC_FEATURE_SEND_TO_ZULIP` |
| `rooms` | `NEXT_PUBLIC_FEATURE_ROOMS` |
### Setting Feature Flags
Feature flags are controlled via environment variables using the pattern `NEXT_PUBLIC_FEATURE_{FEATURE_NAME}` where `{FEATURE_NAME}` is the SCREAMING_SNAKE_CASE version of the feature name.
**Examples:**
```bash
# Enable user authentication requirement
NEXT_PUBLIC_FEATURE_REQUIRE_LOGIN=true
# Disable browse functionality
NEXT_PUBLIC_FEATURE_BROWSE=false
# Enable Zulip integration
NEXT_PUBLIC_FEATURE_SEND_TO_ZULIP=true
```

3
compose.yml
View File

@@ -6,6 +6,7 @@ services:
- 1250:1250
volumes:
- ./server/:/app/
- /app/.venv
env_file:
- ./server/.env
environment:
@@ -16,6 +17,7 @@ services:
context: server
volumes:
- ./server/:/app/
- /app/.venv
env_file:
- ./server/.env
environment:
@@ -26,6 +28,7 @@ services:
context: server
volumes:
- ./server/:/app/
- /app/.venv
env_file:
- ./server/.env
environment:

194
server/docs/gpu/api-transcription.md Normal file
View File

@@ -0,0 +1,194 @@
## Reflector GPU Transcription API (Specification)
This document defines the Reflector GPU transcription API that all implementations must adhere to. Current implementations include NVIDIA Parakeet (NeMo) and Whisper (faster-whisper), both deployed on Modal.com. The API surface and response shapes are OpenAI/Whisper-compatible, so clients can switch implementations by changing only the base URL.
### Base URL and Authentication
- Example base URLs (Modal web endpoints):
- Parakeet: `https://<account>--reflector-transcriber-parakeet-web.modal.run`
- Whisper: `https://<account>--reflector-transcriber-web.modal.run`
- All endpoints are served under `/v1` and require a Bearer token:
```
Authorization: Bearer <REFLECTOR_GPU_APIKEY>
```
Note: To switch implementations, deploy the desired variant and point `TRANSCRIPT_URL` to its base URL. The API is identical.
### Supported file types
`mp3, mp4, mpeg, mpga, m4a, wav, webm`
### Models and languages
- Parakeet (NVIDIA NeMo): default `nvidia/parakeet-tdt-0.6b-v2`
- Language support: only `en`. Other languages return HTTP 400.
- Whisper (faster-whisper): default `large-v2` (or deployment-specific)
- Language support: multilingual (per Whisper model capabilities).
Note: The `model` parameter is accepted by all implementations for interface parity. Some backends may treat it as informational.
### Endpoints
#### POST /v1/audio/transcriptions
Transcribe one or more uploaded audio files.
Request: multipart/form-data
- `file` (File) — optional. Single file to transcribe.
- `files` (File[]) — optional. One or more files to transcribe.
- `model` (string) — optional. Defaults to the implementation-specific model (see above).
- `language` (string) — optional, defaults to `en`.
- Parakeet: only `en` is accepted; other values return HTTP 400
- Whisper: model-dependent; typically multilingual
- `batch` (boolean) — optional, defaults to `false`.
Notes:
- Provide either `file` or `files`, not both. If neither is provided, HTTP 400.
- `batch` requires `files`; using `batch=true` without `files` returns HTTP 400.
- Response shape for multiple files is the same regardless of `batch`.
- Files sent to this endpoint are processed in a single pass (no VAD/chunking). This is intended for short clips (roughly ≤ 30s; depends on GPU memory/model). For longer audio, prefer `/v1/audio/transcriptions-from-url` which supports VAD-based chunking.
Responses
Single file response:
```json
{
"text": "transcribed text",
"words": [
{ "word": "hello", "start": 0.0, "end": 0.5 },
{ "word": "world", "start": 0.5, "end": 1.0 }
],
"filename": "audio.mp3"
}
```
Multiple files response:
```json
{
"results": [
{"filename": "a1.mp3", "text": "...", "words": [...]},
{"filename": "a2.mp3", "text": "...", "words": [...]}]
}
```
Notes:
- Word objects always include keys: `word`, `start`, `end`.
- Some implementations may include a trailing space in `word` to match Whisper tokenization behavior; clients should trim if needed.
Example curl (single file):
```bash
curl -X POST \
-H "Authorization: Bearer $REFLECTOR_GPU_APIKEY" \
-F "file=@/path/to/audio.mp3" \
-F "language=en" \
"$BASE_URL/v1/audio/transcriptions"
```
Example curl (multiple files, batch):
```bash
curl -X POST \
-H "Authorization: Bearer $REFLECTOR_GPU_APIKEY" \
-F "files=@/path/a1.mp3" -F "files=@/path/a2.mp3" \
-F "batch=true" -F "language=en" \
"$BASE_URL/v1/audio/transcriptions"
```
#### POST /v1/audio/transcriptions-from-url
Transcribe a single remote audio file by URL.
Request: application/json
Body parameters:
- `audio_file_url` (string) — required. URL of the audio file to transcribe.
- `model` (string) — optional. Defaults to the implementation-specific model (see above).
- `language` (string) — optional, defaults to `en`. Parakeet only accepts `en`.
- `timestamp_offset` (number) — optional, defaults to `0.0`. Added to each word's `start`/`end` in the response.
```json
{
"audio_file_url": "https://example.com/audio.mp3",
"model": "nvidia/parakeet-tdt-0.6b-v2",
"language": "en",
"timestamp_offset": 0.0
}
```
Response:
```json
{
"text": "transcribed text",
"words": [
{ "word": "hello", "start": 10.0, "end": 10.5 },
{ "word": "world", "start": 10.5, "end": 11.0 }
]
}
```
Notes:
- `timestamp_offset` is added to each word’s `start`/`end` in the response.
- Implementations may perform VAD-based chunking and batching for long-form audio; word timings are adjusted accordingly.
Example curl:
```bash
curl -X POST \
-H "Authorization: Bearer $REFLECTOR_GPU_APIKEY" \
-H "Content-Type: application/json" \
-d '{
"audio_file_url": "https://example.com/audio.mp3",
"language": "en",
"timestamp_offset": 0
}' \
"$BASE_URL/v1/audio/transcriptions-from-url"
```
### Error handling
- 400 Bad Request
- Parakeet: `language` other than `en`
- Missing required parameters (`file`/`files` for upload; `audio_file_url` for URL endpoint)
- Unsupported file extension
- 401 Unauthorized
- Missing or invalid Bearer token
- 404 Not Found
- `audio_file_url` does not exist
### Implementation details
- GPUs: A10G for small-file/live, L40S for large-file URL transcription (subject to deployment)
- VAD chunking and segment batching; word timings adjusted and overlapping ends constrained
- Pads very short segments (< 0.5s) to avoid model crashes on some backends
### Server configuration (Reflector API)
Set the Reflector server to use the Modal backend and point `TRANSCRIPT_URL` to your chosen deployment:
```
TRANSCRIPT_BACKEND=modal
TRANSCRIPT_URL=https://<account>--reflector-transcriber-parakeet-web.modal.run
TRANSCRIPT_MODAL_API_KEY=<REFLECTOR_GPU_APIKEY>
```
### Conformance tests
Use the pytest-based conformance tests to validate any new implementation (including self-hosted) against this spec:
```
TRANSCRIPT_URL=https://<your-deployment-base> \
TRANSCRIPT_MODAL_API_KEY=your-api-key \
uv run -m pytest -m gpu_modal --no-cov server/tests/test_gpu_modal_transcript.py
```

569
server/gpu/modal_deployments/reflector_transcriber.py
View File

@@ -1,41 +1,78 @@
import os
import tempfile
import sys
import threading
import uuid
from typing import Generator, Mapping, NamedTuple, NewType, TypedDict
from urllib.parse import urlparse
import modal
from pydantic import BaseModel
MODELS_DIR = "/models"
MODEL_NAME = "large-v2"
MODEL_COMPUTE_TYPE: str = "float16"
MODEL_NUM_WORKERS: int = 1
MINUTES = 60 # seconds
SAMPLERATE = 16000
UPLOADS_PATH = "/uploads"
CACHE_PATH = "/models"
SUPPORTED_FILE_EXTENSIONS = ["mp3", "mp4", "mpeg", "mpga", "m4a", "wav", "webm"]
VAD_CONFIG = {
"batch_max_duration": 30.0,
"silence_padding": 0.5,
"window_size": 512,
}
volume = modal.Volume.from_name("models", create_if_missing=True)
WhisperUniqFilename = NewType("WhisperUniqFilename", str)
AudioFileExtension = NewType("AudioFileExtension", str)
app = modal.App("reflector-transcriber")
model_cache = modal.Volume.from_name("models", create_if_missing=True)
upload_volume = modal.Volume.from_name("whisper-uploads", create_if_missing=True)
class TimeSegment(NamedTuple):
"""Represents a time segment with start and end times."""
start: float
end: float
class AudioSegment(NamedTuple):
"""Represents an audio segment with timing and audio data."""
start: float
end: float
audio: any
class TranscriptResult(NamedTuple):
"""Represents a transcription result with text and word timings."""
text: str
words: list["WordTiming"]
class WordTiming(TypedDict):
"""Represents a word with its timing information."""
word: str
start: float
end: float
def download_model():
from faster_whisper import download_model
volume.reload()
model_cache.reload()
download_model(MODEL_NAME, cache_dir=MODELS_DIR)
download_model(MODEL_NAME, cache_dir=CACHE_PATH)
volume.commit()
model_cache.commit()
image = (
modal.Image.debian_slim(python_version="3.12")
.pip_install(
"huggingface_hub==0.27.1",
"hf-transfer==0.1.9",
"torch==2.5.1",
"faster-whisper==1.1.1",
)
.env(
{
"HF_HUB_ENABLE_HF_TRANSFER": "1",
@@ -45,19 +82,98 @@ image = (
),
}
)
.run_function(download_model, volumes={MODELS_DIR: volume})
.apt_install("ffmpeg")
.pip_install(
"huggingface_hub==0.27.1",
"hf-transfer==0.1.9",
"torch==2.5.1",
"faster-whisper==1.1.1",
"fastapi==0.115.12",
"requests",
"librosa==0.10.1",
"numpy<2",
"silero-vad==5.1.0",
)
.run_function(download_model, volumes={CACHE_PATH: model_cache})
)
def detect_audio_format(url: str, headers: Mapping[str, str]) -> AudioFileExtension:
parsed_url = urlparse(url)
url_path = parsed_url.path
for ext in SUPPORTED_FILE_EXTENSIONS:
if url_path.lower().endswith(f".{ext}"):
return AudioFileExtension(ext)
content_type = headers.get("content-type", "").lower()
if "audio/mpeg" in content_type or "audio/mp3" in content_type:
return AudioFileExtension("mp3")
if "audio/wav" in content_type:
return AudioFileExtension("wav")
if "audio/mp4" in content_type:
return AudioFileExtension("mp4")
raise ValueError(
f"Unsupported audio format for URL: {url}. "
f"Supported extensions: {', '.join(SUPPORTED_FILE_EXTENSIONS)}"
)
def download_audio_to_volume(
audio_file_url: str,
) -> tuple[WhisperUniqFilename, AudioFileExtension]:
import requests
from fastapi import HTTPException
response = requests.head(audio_file_url, allow_redirects=True)
if response.status_code == 404:
raise HTTPException(status_code=404, detail="Audio file not found")
response = requests.get(audio_file_url, allow_redirects=True)
response.raise_for_status()
audio_suffix = detect_audio_format(audio_file_url, response.headers)
unique_filename = WhisperUniqFilename(f"{uuid.uuid4()}.{audio_suffix}")
file_path = f"{UPLOADS_PATH}/{unique_filename}"
with open(file_path, "wb") as f:
f.write(response.content)
upload_volume.commit()
return unique_filename, audio_suffix
def pad_audio(audio_array, sample_rate: int = SAMPLERATE):
"""Add 0.5s of silence if audio is shorter than the silence_padding window.
Whisper does not require this strictly, but aligning behavior with Parakeet
avoids edge-case crashes on extremely short inputs and makes comparisons easier.
"""
import numpy as np
audio_duration = len(audio_array) / sample_rate
if audio_duration < VAD_CONFIG["silence_padding"]:
silence_samples = int(sample_rate * VAD_CONFIG["silence_padding"])
silence = np.zeros(silence_samples, dtype=np.float32)
return np.concatenate([audio_array, silence])
return audio_array
@app.cls(
gpu="A10G",
timeout=5 * MINUTES,
scaledown_window=5 * MINUTES,
allow_concurrent_inputs=6,
image=image,
volumes={MODELS_DIR: volume},
volumes={CACHE_PATH: model_cache, UPLOADS_PATH: upload_volume},
)
class Transcriber:
@modal.concurrent(max_inputs=10)
class TranscriberWhisperLive:
"""Live transcriber class for small audio segments (A10G).
Mirrors the Parakeet live class API but uses Faster-Whisper under the hood.
"""
@modal.enter()
def enter(self):
import faster_whisper
@@ -71,23 +187,200 @@ class Transcriber:
device=self.device,
compute_type=MODEL_COMPUTE_TYPE,
num_workers=MODEL_NUM_WORKERS,
download_root=MODELS_DIR,
download_root=CACHE_PATH,
local_files_only=True,
)
print(f"Model is on device: {self.device}")
@modal.method()
def transcribe_segment(
self,
audio_data: str,
audio_suffix: str,
language: str,
filename: str,
language: str = "en",
):
with tempfile.NamedTemporaryFile("wb+", suffix=f".{audio_suffix}") as fp:
fp.write(audio_data)
"""Transcribe a single uploaded audio file by filename."""
upload_volume.reload()
file_path = f"{UPLOADS_PATH}/{filename}"
if not os.path.exists(file_path):
raise FileNotFoundError(f"File not found: {file_path}")
with self.lock:
with NoStdStreams():
segments, _ = self.model.transcribe(
file_path,
language=language,
beam_size=5,
word_timestamps=True,
vad_filter=True,
vad_parameters={"min_silence_duration_ms": 500},
)
segments = list(segments)
text = "".join(segment.text for segment in segments).strip()
words = [
{
"word": word.word,
"start": round(float(word.start), 2),
"end": round(float(word.end), 2),
}
for segment in segments
for word in segment.words
]
return {"text": text, "words": words}
@modal.method()
def transcribe_batch(
self,
filenames: list[str],
language: str = "en",
):
"""Transcribe multiple uploaded audio files and return per-file results."""
upload_volume.reload()
results = []
for filename in filenames:
file_path = f"{UPLOADS_PATH}/{filename}"
if not os.path.exists(file_path):
raise FileNotFoundError(f"Batch file not found: {file_path}")
with self.lock:
with NoStdStreams():
segments, _ = self.model.transcribe(
file_path,
language=language,
beam_size=5,
word_timestamps=True,
vad_filter=True,
vad_parameters={"min_silence_duration_ms": 500},
)
segments = list(segments)
text = "".join(seg.text for seg in segments).strip()
words = [
{
"word": w.word,
"start": round(float(w.start), 2),
"end": round(float(w.end), 2),
}
for seg in segments
for w in seg.words
]
results.append(
{
"filename": filename,
"text": text,
"words": words,
}
)
return results
@app.cls(
gpu="L40S",
timeout=15 * MINUTES,
image=image,
volumes={CACHE_PATH: model_cache, UPLOADS_PATH: upload_volume},
)
class TranscriberWhisperFile:
"""File transcriber for larger/longer audio, using VAD-driven batching (L40S)."""
@modal.enter()
def enter(self):
import faster_whisper
import torch
from silero_vad import load_silero_vad
self.lock = threading.Lock()
self.use_gpu = torch.cuda.is_available()
self.device = "cuda" if self.use_gpu else "cpu"
self.model = faster_whisper.WhisperModel(
MODEL_NAME,
device=self.device,
compute_type=MODEL_COMPUTE_TYPE,
num_workers=MODEL_NUM_WORKERS,
download_root=CACHE_PATH,
local_files_only=True,
)
self.vad_model = load_silero_vad(onnx=False)
@modal.method()
def transcribe_segment(
self, filename: str, timestamp_offset: float = 0.0, language: str = "en"
):
import librosa
import numpy as np
from silero_vad import VADIterator
def vad_segments(
audio_array,
sample_rate: int = SAMPLERATE,
window_size: int = VAD_CONFIG["window_size"],
) -> Generator[TimeSegment, None, None]:
"""Generate speech segments as TimeSegment using Silero VAD."""
iterator = VADIterator(self.vad_model, sampling_rate=sample_rate)
start = None
for i in range(0, len(audio_array), window_size):
chunk = audio_array[i : i + window_size]
if len(chunk) < window_size:
chunk = np.pad(
chunk, (0, window_size - len(chunk)), mode="constant"
)
speech = iterator(chunk)
if not speech:
continue
if "start" in speech:
start = speech["start"]
continue
if "end" in speech and start is not None:
end = speech["end"]
yield TimeSegment(
start / float(SAMPLERATE), end / float(SAMPLERATE)
)
start = None
iterator.reset_states()
upload_volume.reload()
file_path = f"{UPLOADS_PATH}/{filename}"
if not os.path.exists(file_path):
raise FileNotFoundError(f"File not found: {file_path}")
audio_array, _sr = librosa.load(file_path, sr=SAMPLERATE, mono=True)
# Batch segments up to ~30s windows by merging contiguous VAD segments
merged_batches: list[TimeSegment] = []
batch_start = None
batch_end = None
max_duration = VAD_CONFIG["batch_max_duration"]
for segment in vad_segments(audio_array):
seg_start, seg_end = segment.start, segment.end
if batch_start is None:
batch_start, batch_end = seg_start, seg_end
continue
if seg_end - batch_start <= max_duration:
batch_end = seg_end
else:
merged_batches.append(TimeSegment(batch_start, batch_end))
batch_start, batch_end = seg_start, seg_end
if batch_start is not None and batch_end is not None:
merged_batches.append(TimeSegment(batch_start, batch_end))
all_text = []
all_words = []
for segment in merged_batches:
start_time, end_time = segment.start, segment.end
s_idx = int(start_time * SAMPLERATE)
e_idx = int(end_time * SAMPLERATE)
segment = audio_array[s_idx:e_idx]
segment = pad_audio(segment, SAMPLERATE)
with self.lock:
segments, _ = self.model.transcribe(
fp.name,
segment,
language=language,
beam_size=5,
word_timestamps=True,
@@ -96,66 +389,220 @@ class Transcriber:
)
segments = list(segments)
text = "".join(segment.text for segment in segments)
text = "".join(seg.text for seg in segments).strip()
words = [
{"word": word.word, "start": word.start, "end": word.end}
for segment in segments
for word in segment.words
{
"word": w.word,
"start": round(float(w.start) + start_time + timestamp_offset, 2),
"end": round(float(w.end) + start_time + timestamp_offset, 2),
}
for seg in segments
for w in seg.words
]
if text:
all_text.append(text)
all_words.extend(words)
return {"text": text, "words": words}
return {"text": " ".join(all_text), "words": all_words}
def detect_audio_format(url: str, headers: dict) -> str:
from urllib.parse import urlparse
from fastapi import HTTPException
url_path = urlparse(url).path
for ext in SUPPORTED_FILE_EXTENSIONS:
if url_path.lower().endswith(f".{ext}"):
return ext
content_type = headers.get("content-type", "").lower()
if "audio/mpeg" in content_type or "audio/mp3" in content_type:
return "mp3"
if "audio/wav" in content_type:
return "wav"
if "audio/mp4" in content_type:
return "mp4"
raise HTTPException(
status_code=400,
detail=(
f"Unsupported audio format for URL. Supported extensions: {', '.join(SUPPORTED_FILE_EXTENSIONS)}"
),
)
def download_audio_to_volume(audio_file_url: str) -> tuple[str, str]:
import requests
from fastapi import HTTPException
response = requests.head(audio_file_url, allow_redirects=True)
if response.status_code == 404:
raise HTTPException(status_code=404, detail="Audio file not found")
response = requests.get(audio_file_url, allow_redirects=True)
response.raise_for_status()
audio_suffix = detect_audio_format(audio_file_url, response.headers)
unique_filename = f"{uuid.uuid4()}.{audio_suffix}"
file_path = f"{UPLOADS_PATH}/{unique_filename}"
with open(file_path, "wb") as f:
f.write(response.content)
upload_volume.commit()
return unique_filename, audio_suffix
@app.function(
scaledown_window=60,
timeout=60,
allow_concurrent_inputs=40,
timeout=600,
secrets=[
modal.Secret.from_name("reflector-gpu"),
],
volumes={MODELS_DIR: volume},
volumes={CACHE_PATH: model_cache, UPLOADS_PATH: upload_volume},
image=image,
)
@modal.concurrent(max_inputs=40)
@modal.asgi_app()
def web():
from fastapi import Body, Depends, FastAPI, HTTPException, UploadFile, status
from fastapi import (
Body,
Depends,
FastAPI,
Form,
HTTPException,
UploadFile,
status,
)
from fastapi.security import OAuth2PasswordBearer
from typing_extensions import Annotated
transcriber = Transcriber()
transcriber_live = TranscriberWhisperLive()
transcriber_file = TranscriberWhisperFile()
app = FastAPI()
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")
supported_file_types = ["mp3", "mp4", "mpeg", "mpga", "m4a", "wav", "webm"]
def apikey_auth(apikey: str = Depends(oauth2_scheme)):
if apikey != os.environ["REFLECTOR_GPU_APIKEY"]:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid API key",
headers={"WWW-Authenticate": "Bearer"},
)
if apikey == os.environ["REFLECTOR_GPU_APIKEY"]:
return
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid API key",
headers={"WWW-Authenticate": "Bearer"},
)
class TranscriptResponse(BaseModel):
result: dict
class TranscriptResponse(dict):
pass
@app.post("/v1/audio/transcriptions", dependencies=[Depends(apikey_auth)])
def transcribe(
file: UploadFile,
model: str = "whisper-1",
language: Annotated[str, Body(...)] = "en",
) -> TranscriptResponse:
audio_data = file.file.read()
audio_suffix = file.filename.split(".")[-1]
assert audio_suffix in supported_file_types
file: UploadFile = None,
files: list[UploadFile] | None = None,
model: str = Form(MODEL_NAME),
language: str = Form("en"),
batch: bool = Form(False),
):
if not file and not files:
raise HTTPException(
status_code=400, detail="Either 'file' or 'files' parameter is required"
)
if batch and not files:
raise HTTPException(
status_code=400, detail="Batch transcription requires 'files'"
)
func = transcriber.transcribe_segment.spawn(
audio_data=audio_data,
audio_suffix=audio_suffix,
language=language,
)
result = func.get()
return result
upload_files = [file] if file else files
uploaded_filenames: list[str] = []
for upload_file in upload_files:
audio_suffix = upload_file.filename.split(".")[-1]
if audio_suffix not in SUPPORTED_FILE_EXTENSIONS:
raise HTTPException(
status_code=400,
detail=(
f"Unsupported audio format. Supported extensions: {', '.join(SUPPORTED_FILE_EXTENSIONS)}"
),
)
unique_filename = f"{uuid.uuid4()}.{audio_suffix}"
file_path = f"{UPLOADS_PATH}/{unique_filename}"
with open(file_path, "wb") as f:
content = upload_file.file.read()
f.write(content)
uploaded_filenames.append(unique_filename)
upload_volume.commit()
try:
if batch and len(upload_files) > 1:
func = transcriber_live.transcribe_batch.spawn(
filenames=uploaded_filenames,
language=language,
)
results = func.get()
return {"results": results}
results = []
for filename in uploaded_filenames:
func = transcriber_live.transcribe_segment.spawn(
filename=filename,
language=language,
)
result = func.get()
result["filename"] = filename
results.append(result)
return {"results": results} if len(results) > 1 else results[0]
finally:
for filename in uploaded_filenames:
try:
file_path = f"{UPLOADS_PATH}/{filename}"
os.remove(file_path)
except Exception:
pass
upload_volume.commit()
@app.post("/v1/audio/transcriptions-from-url", dependencies=[Depends(apikey_auth)])
def transcribe_from_url(
audio_file_url: str = Body(
..., description="URL of the audio file to transcribe"
),
model: str = Body(MODEL_NAME),
language: str = Body("en"),
timestamp_offset: float = Body(0.0),
):
unique_filename, _audio_suffix = download_audio_to_volume(audio_file_url)
try:
func = transcriber_file.transcribe_segment.spawn(
filename=unique_filename,
timestamp_offset=timestamp_offset,
language=language,
)
result = func.get()
return result
finally:
try:
file_path = f"{UPLOADS_PATH}/{unique_filename}"
os.remove(file_path)
upload_volume.commit()
except Exception:
pass
return app
class NoStdStreams:
def __init__(self):
self.devnull = open(os.devnull, "w")
def __enter__(self):
self._stdout, self._stderr = sys.stdout, sys.stderr
self._stdout.flush()
self._stderr.flush()
sys.stdout, sys.stderr = self.devnull, self.devnull
def __exit__(self, exc_type, exc_value, traceback):
sys.stdout, sys.stderr = self._stdout, self._stderr
self.devnull.close()

36
server/migrations/versions/0ce521cda2ee_remove_user_id_from_meeting_table.py Normal file
View File

@@ -0,0 +1,36 @@
"""remove user_id from meeting table
Revision ID: 0ce521cda2ee
Revises: 6dec9fb5b46c
Create Date: 2025-09-10 12:40:55.688899
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "0ce521cda2ee"
down_revision: Union[str, None] = "6dec9fb5b46c"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.drop_column("user_id")
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.add_column(
sa.Column("user_id", sa.VARCHAR(), autoincrement=False, nullable=True)
)
# ### end Alembic commands ###

32
server/migrations/versions/2ae3db106d4e_clean_up_orphaned_room_id_references_in_.py Normal file
View File

@@ -0,0 +1,32 @@
"""clean up orphaned room_id references in meeting table
Revision ID: 2ae3db106d4e
Revises: def1b5867d4c
Create Date: 2025-09-11 10:35:15.759967
"""
from typing import Sequence, Union
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "2ae3db106d4e"
down_revision: Union[str, None] = "def1b5867d4c"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Set room_id to NULL for meetings that reference non-existent rooms
op.execute("""
UPDATE meeting
SET room_id = NULL
WHERE room_id IS NOT NULL
AND room_id NOT IN (SELECT id FROM room WHERE id IS NOT NULL)
""")
def downgrade() -> None:
# Cannot restore orphaned references - no operation needed
pass

38
server/migrations/versions/6dec9fb5b46c_make_meeting_room_id_required_and_add_.py Normal file
View File

@@ -0,0 +1,38 @@
"""make meeting room_id required and add foreign key
Revision ID: 6dec9fb5b46c
Revises: 61882a919591
Create Date: 2025-09-10 10:47:06.006819
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "6dec9fb5b46c"
down_revision: Union[str, None] = "61882a919591"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.alter_column("room_id", existing_type=sa.VARCHAR(), nullable=False)
batch_op.create_foreign_key(
None, "room", ["room_id"], ["id"], ondelete="CASCADE"
)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.drop_constraint("meeting_room_id_fkey", type_="foreignkey")
batch_op.alter_column("room_id", existing_type=sa.VARCHAR(), nullable=True)
# ### end Alembic commands ###

34
server/migrations/versions/def1b5867d4c_make_meeting_room_id_nullable_but_keep_.py Normal file
View File

@@ -0,0 +1,34 @@
"""make meeting room_id nullable but keep foreign key
Revision ID: def1b5867d4c
Revises: 0ce521cda2ee
Create Date: 2025-09-11 09:42:18.697264
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "def1b5867d4c"
down_revision: Union[str, None] = "0ce521cda2ee"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.alter_column("room_id", existing_type=sa.VARCHAR(), nullable=True)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("meeting", schema=None) as batch_op:
batch_op.alter_column("room_id", existing_type=sa.VARCHAR(), nullable=False)
# ### end Alembic commands ###

52
server/reflector/db/meetings.py
View File

@@ -2,7 +2,6 @@ from datetime import datetime, timezone
from typing import Any, Dict, List, Literal
import sqlalchemy as sa
from fastapi import HTTPException
from pydantic import BaseModel, Field
from reflector.db import get_database, metadata
@@ -18,8 +17,12 @@ meetings = sa.Table(
sa.Column("host_room_url", sa.String),
sa.Column("start_date", sa.DateTime(timezone=True)),
sa.Column("end_date", sa.DateTime(timezone=True)),
sa.Column("user_id", sa.String),
sa.Column("room_id", sa.String),
sa.Column(
"room_id",
sa.String,
sa.ForeignKey("room.id", ondelete="CASCADE"),
nullable=True,
),
sa.Column("is_locked", sa.Boolean, nullable=False, server_default=sa.false()),
sa.Column("room_mode", sa.String, nullable=False, server_default="normal"),
sa.Column("recording_type", sa.String, nullable=False, server_default="cloud"),
@@ -83,8 +86,7 @@ class Meeting(BaseModel):
host_room_url: str
start_date: datetime
end_date: datetime
user_id: str | None = None
room_id: str | None = None
room_id: str | None
is_locked: bool = False
room_mode: Literal["normal", "group"] = "normal"
recording_type: Literal["none", "local", "cloud"] = "cloud"
@@ -105,12 +107,8 @@ class MeetingController:
host_room_url: str,
start_date: datetime,
end_date: datetime,
user_id: str,
room: Room,
):
"""
Create a new meeting
"""
meeting = Meeting(
id=id,
room_name=room_name,
@@ -118,7 +116,6 @@ class MeetingController:
host_room_url=host_room_url,
start_date=start_date,
end_date=end_date,
user_id=user_id,
room_id=room.id,
is_locked=room.is_locked,
room_mode=room.room_mode,
@@ -131,19 +128,13 @@ class MeetingController:
return meeting
async def get_all_active(self) -> list[Meeting]:
"""
Get active meetings.
"""
query = meetings.select().where(meetings.c.is_active)
return await get_database().fetch_all(query)
async def get_by_room_name(
self,
room_name: str,
) -> Meeting:
"""
Get a meeting by room name.
"""
) -> Meeting | None:
query = meetings.select().where(meetings.c.room_name == room_name)
result = await get_database().fetch_one(query)
if not result:
@@ -151,10 +142,7 @@ class MeetingController:
return Meeting(**result)
async def get_active(self, room: Room, current_time: datetime) -> Meeting:
"""
Get latest active meeting for a room.
"""
async def get_active(self, room: Room, current_time: datetime) -> Meeting | None:
end_date = getattr(meetings.c, "end_date")
query = (
meetings.select()
@@ -174,32 +162,12 @@ class MeetingController:
return Meeting(**result)
async def get_by_id(self, meeting_id: str, **kwargs) -> Meeting | None:
"""
Get a meeting by id
"""
query = meetings.select().where(meetings.c.id == meeting_id)
result = await get_database().fetch_one(query)
if not result:
return None
return Meeting(**result)
async def get_by_id_for_http(self, meeting_id: str, user_id: str | None) -> Meeting:
"""
Get a meeting by ID for HTTP request.
If not found, it will raise a 404 error.
"""
query = meetings.select().where(meetings.c.id == meeting_id)
result = await get_database().fetch_one(query)
if not result:
raise HTTPException(status_code=404, detail="Meeting not found")
meeting = Meeting(**result)
if result["user_id"] != user_id:
meeting.host_room_url = ""
return meeting
async def update_meeting(self, meeting_id: str, **kwargs):
query = meetings.update().where(meetings.c.id == meeting_id).values(**kwargs)
await get_database().execute(query)
@@ -286,7 +254,7 @@ class MeetingConsentController:
result = await get_database().fetch_one(query)
if result is None:
return None
return MeetingConsent(**result) if result else None
return MeetingConsent(**result)
async def upsert(self, consent: MeetingConsent) -> MeetingConsent:
"""Create new consent or update existing one for authenticated users"""

4
server/reflector/db/search.py
View File

@@ -23,7 +23,7 @@ from pydantic import (
from reflector.db import get_database
from reflector.db.rooms import rooms
from reflector.db.transcripts import SourceKind, transcripts
from reflector.db.transcripts import SourceKind, TranscriptStatus, transcripts
from reflector.db.utils import is_postgresql
from reflector.logger import logger
from reflector.utils.string import NonEmptyString, try_parse_non_empty_string
@@ -161,7 +161,7 @@ class SearchResult(BaseModel):
room_name: str | None = None
source_kind: SourceKind
created_at: datetime
status: str = Field(..., min_length=1)
status: TranscriptStatus = Field(..., min_length=1)
rank: float = Field(..., ge=0, le=1)
duration: NonNegativeFloat | None = Field(..., description="Duration in seconds")
search_snippets: list[str] = Field(

46
server/reflector/pipelines/main_file_pipeline.py
View File

@@ -12,7 +12,7 @@ from pathlib import Path
import av
import structlog
from celery import shared_task
from celery import chain, shared_task
from reflector.asynctask import asynctask
from reflector.db.rooms import rooms_controller
@@ -26,6 +26,8 @@ from reflector.logger import logger
from reflector.pipelines.main_live_pipeline import (
PipelineMainBase,
broadcast_to_sockets,
task_cleanup_consent,
task_pipeline_post_to_zulip,
)
from reflector.processors import (
AudioFileWriterProcessor,
@@ -379,6 +381,28 @@ class PipelineMainFile(PipelineMainBase):
await processor.flush()
@shared_task
@asynctask
async def task_send_webhook_if_needed(*, transcript_id: str):
"""Send webhook if this is a room recording with webhook configured"""
transcript = await transcripts_controller.get_by_id(transcript_id)
if not transcript:
return
if transcript.source_kind == SourceKind.ROOM and transcript.room_id:
room = await rooms_controller.get_by_id(transcript.room_id)
if room and room.webhook_url:
logger.info(
"Dispatching webhook",
transcript_id=transcript_id,
room_id=room.id,
webhook_url=room.webhook_url,
)
send_transcript_webhook.delay(
transcript_id, room.id, event_id=uuid.uuid4().hex
)
@shared_task
@asynctask
async def task_pipeline_file_process(*, transcript_id: str):
@@ -406,16 +430,10 @@ async def task_pipeline_file_process(*, transcript_id: str):
await pipeline.set_status(transcript_id, "error")
raise
# Trigger webhook if this is a room recording with webhook configured
if transcript.source_kind == SourceKind.ROOM and transcript.room_id:
room = await rooms_controller.get_by_id(transcript.room_id)
if room and room.webhook_url:
logger.info(
"Dispatching webhook task",
transcript_id=transcript_id,
room_id=room.id,
webhook_url=room.webhook_url,
)
send_transcript_webhook.delay(
transcript_id, room.id, event_id=uuid.uuid4().hex
)
# Run post-processing chain: consent cleanup -> zulip -> webhook
post_chain = chain(
task_cleanup_consent.si(transcript_id=transcript_id),
task_pipeline_post_to_zulip.si(transcript_id=transcript_id),
task_send_webhook_if_needed.si(transcript_id=transcript_id),
)
post_chain.delay()

1
server/reflector/processors/file_diarization_modal.py
View File

@@ -47,6 +47,7 @@ class FileDiarizationModalProcessor(FileDiarizationProcessor):
"audio_file_url": data.audio_url,
"timestamp": 0,
},
follow_redirects=True,
)
response.raise_for_status()
diarization_data = response.json()["diarization"]

1
server/reflector/processors/file_transcript_modal.py
View File

@@ -54,6 +54,7 @@ class FileTranscriptModalProcessor(FileTranscriptProcessor):
"language": data.language,
"batch": True,
},
follow_redirects=True,
)
response.raise_for_status()
result = response.json()

4
server/reflector/settings.py
View File

@@ -1,6 +1,8 @@
from pydantic.types import PositiveInt
from pydantic_settings import BaseSettings, SettingsConfigDict
from reflector.utils.string import NonEmptyString
class Settings(BaseSettings):
model_config = SettingsConfigDict(
@@ -120,7 +122,7 @@ class Settings(BaseSettings):
# Whereby integration
WHEREBY_API_URL: str = "https://api.whereby.dev/v1"
WHEREBY_API_KEY: str | None = None
WHEREBY_API_KEY: NonEmptyString | None = None
WHEREBY_WEBHOOK_SECRET: str | None = None
AWS_WHEREBY_ACCESS_KEY_ID: str | None = None
AWS_WHEREBY_ACCESS_KEY_SECRET: str | None = None

7
server/reflector/utils/string.py
View File

@@ -10,8 +10,11 @@ NonEmptyString = Annotated[
non_empty_string_adapter = TypeAdapter(NonEmptyString)
def parse_non_empty_string(s: str) -> NonEmptyString:
return non_empty_string_adapter.validate_python(s)
def parse_non_empty_string(s: str, error: str | None = None) -> NonEmptyString:
try:
return non_empty_string_adapter.validate_python(s)
except Exception as e:
raise ValueError(f"{e}: {error}" if error else e) from e
def try_parse_non_empty_string(s: str) -> NonEmptyString | None:

4
server/reflector/views/rooms.py
View File

@@ -241,7 +241,9 @@ async def rooms_create_meeting(
)
if meeting is None:
logger.error(
"Meeting disappeared after race condition for room %s", room.name
"Meeting disappeared after race condition for room %s",
room.name,
exc_info=True,
)
raise HTTPException(
status_code=503, detail="Unable to join meeting - please try again"

8
server/reflector/views/transcripts.py
View File

@@ -27,6 +27,7 @@ from reflector.db.search import (
from reflector.db.transcripts import (
SourceKind,
TranscriptParticipant,
TranscriptStatus,
TranscriptTopic,
transcripts_controller,
)
@@ -63,7 +64,7 @@ class GetTranscriptMinimal(BaseModel):
id: str
user_id: str | None
name: str
status: str
status: TranscriptStatus
locked: bool
duration: float
title: str | None
@@ -96,6 +97,7 @@ class CreateTranscript(BaseModel):
name: str
source_language: str = Field("en")
target_language: str = Field("en")
source_kind: SourceKind | None = None
class UpdateTranscript(BaseModel):
@@ -213,7 +215,7 @@ async def transcripts_create(
user_id = user["sub"] if user else None
return await transcripts_controller.add(
info.name,
source_kind=SourceKind.LIVE,
source_kind=info.source_kind or SourceKind.LIVE,
source_language=info.source_language,
target_language=info.target_language,
user_id=user_id,
@@ -348,8 +350,6 @@ async def transcript_update(
transcript = await transcripts_controller.get_by_id_for_http(
transcript_id, user_id=user_id
)
if not transcript:
raise HTTPException(status_code=404, detail="Transcript not found")
values = info.dict(exclude_unset=True)
updated_transcript = await transcripts_controller.update(transcript, values)
return updated_transcript

2
server/reflector/views/transcripts_process.py
View File

@@ -34,7 +34,7 @@ async def transcript_process(
)
if task_is_scheduled_or_active(
"reflector.pipelines.main_live_pipeline.task_pipeline_process",
"reflector.pipelines.main_file_pipeline.task_pipeline_file_process",
transcript_id=transcript_id,
):
return ProcessStatus(status="already running")

65
server/reflector/whereby.py
View File

@@ -1,18 +1,60 @@
import logging
from datetime import datetime
import httpx
from reflector.db.rooms import Room
from reflector.settings import settings
from reflector.utils.string import parse_non_empty_string
logger = logging.getLogger(__name__)
def _get_headers():
api_key = parse_non_empty_string(
settings.WHEREBY_API_KEY, "WHEREBY_API_KEY value is required."
)
return {
"Content-Type": "application/json; charset=utf-8",
"Authorization": f"Bearer {api_key}",
}
HEADERS = {
"Content-Type": "application/json; charset=utf-8",
"Authorization": f"Bearer {settings.WHEREBY_API_KEY}",
}
TIMEOUT = 10 # seconds
def _get_whereby_s3_auth():
errors = []
try:
bucket_name = parse_non_empty_string(
settings.RECORDING_STORAGE_AWS_BUCKET_NAME,
"RECORDING_STORAGE_AWS_BUCKET_NAME value is required.",
)
except Exception as e:
errors.append(e)
try:
key_id = parse_non_empty_string(
settings.AWS_WHEREBY_ACCESS_KEY_ID,
"AWS_WHEREBY_ACCESS_KEY_ID value is required.",
)
except Exception as e:
errors.append(e)
try:
key_secret = parse_non_empty_string(
settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
"AWS_WHEREBY_ACCESS_KEY_SECRET value is required.",
)
except Exception as e:
errors.append(e)
if len(errors) > 0:
raise Exception(
f"Failed to get Whereby auth settings: {', '.join(str(e) for e in errors)}"
)
return bucket_name, key_id, key_secret
async def create_meeting(room_name_prefix: str, end_date: datetime, room: Room):
s3_bucket_name, s3_key_id, s3_key_secret = _get_whereby_s3_auth()
data = {
"isLocked": room.is_locked,
"roomNamePrefix": room_name_prefix,
@@ -23,23 +65,26 @@ async def create_meeting(room_name_prefix: str, end_date: datetime, room: Room):
"type": room.recording_type,
"destination": {
"provider": "s3",
"bucket": settings.RECORDING_STORAGE_AWS_BUCKET_NAME,
"accessKeyId": settings.AWS_WHEREBY_ACCESS_KEY_ID,
"accessKeySecret": settings.AWS_WHEREBY_ACCESS_KEY_SECRET,
"bucket": s3_bucket_name,
"accessKeyId": s3_key_id,
"accessKeySecret": s3_key_secret,
"fileFormat": "mp4",
},
"startTrigger": room.recording_trigger,
},
"fields": ["hostRoomUrl"],
}
async with httpx.AsyncClient() as client:
response = await client.post(
f"{settings.WHEREBY_API_URL}/meetings",
headers=HEADERS,
headers=_get_headers(),
json=data,
timeout=TIMEOUT,
)
if response.status_code == 403:
logger.warning(
f"Failed to create meeting: access denied on Whereby: {response.text}"
)
response.raise_for_status()
return response.json()
@@ -48,7 +93,7 @@ async def get_room_sessions(room_name: str):
async with httpx.AsyncClient() as client:
response = await client.get(
f"{settings.WHEREBY_API_URL}/insights/room-sessions?roomName={room_name}",
headers=HEADERS,
headers=_get_headers(),
timeout=TIMEOUT,
)
response.raise_for_status()

2
server/runserver.sh
View File

@@ -2,7 +2,7 @@
if [ "${ENTRYPOINT}" = "server" ]; then
uv run alembic upgrade head
uv run -m reflector.app
uv run uvicorn reflector.app:app --host 0.0.0.0 --port 1250
elif [ "${ENTRYPOINT}" = "worker" ]; then
uv run celery -A reflector.worker.app worker --loglevel=info
elif [ "${ENTRYPOINT}" = "beat" ]; then

2
server/tests/test_cleanup.py
View File

@@ -105,7 +105,6 @@ async def test_cleanup_deletes_associated_meeting_and_recording():
host_room_url="https://example.com/meeting-host",
start_date=old_date,
end_date=old_date + timedelta(hours=1),
user_id=None,
room_id=None,
)
)
@@ -241,7 +240,6 @@ async def test_meeting_consent_cascade_delete():
host_room_url="https://example.com/cascade-test-host",
start_date=datetime.now(timezone.utc),
end_date=datetime.now(timezone.utc) + timedelta(hours=1),
user_id="test-user",
room_id=None,
)
)

3
server/tests/test_gpu_modal_transcript.py
View File

@@ -272,6 +272,9 @@ class TestGPUModalTranscript:
for f in temp_files:
Path(f).unlink(missing_ok=True)
@pytest.mark.skipif(
not "parakeet" in get_model_name(), reason="Parakeet only supports English"
)
def test_transcriptions_error_handling(self):
"""Test error handling for invalid requests."""
url = get_modal_transcript_url()

16
server/tests/test_search.py
View File

@@ -58,7 +58,7 @@ async def test_empty_transcript_title_only_match():
"id": test_id,
"name": "Empty Transcript",
"title": "Empty Meeting",
"status": "completed",
"status": "ended",
"locked": False,
"duration": 0.0,
"created_at": datetime.now(timezone.utc),
@@ -109,7 +109,7 @@ async def test_search_with_long_summary():
"id": test_id,
"name": "Test Long Summary",
"title": "Regular Meeting",
"status": "completed",
"status": "ended",
"locked": False,
"duration": 1800.0,
"created_at": datetime.now(timezone.utc),
@@ -165,7 +165,7 @@ async def test_postgresql_search_with_data():
"id": test_id,
"name": "Test Search Transcript",
"title": "Engineering Planning Meeting Q4 2024",
"status": "completed",
"status": "ended",
"locked": False,
"duration": 1800.0,
"created_at": datetime.now(timezone.utc),
@@ -221,7 +221,7 @@ We need to implement PostgreSQL tsvector for better performance.""",
test_result = next((r for r in results if r.id == test_id), None)
if test_result:
assert test_result.title == "Engineering Planning Meeting Q4 2024"
assert test_result.status == "completed"
assert test_result.status == "ended"
assert test_result.duration == 1800.0
assert 0 <= test_result.rank <= 1, "Rank should be normalized to 0-1"
@@ -268,7 +268,7 @@ def mock_db_result():
"title": "Test Transcript",
"created_at": datetime(2024, 6, 15, tzinfo=timezone.utc),
"duration": 3600.0,
"status": "completed",
"status": "ended",
"user_id": "test-user",
"room_id": "room1",
"source_kind": SourceKind.LIVE,
@@ -433,7 +433,7 @@ class TestSearchResultModel:
room_id="room-456",
source_kind=SourceKind.ROOM,
created_at=datetime(2024, 6, 15, tzinfo=timezone.utc),
status="completed",
status="ended",
rank=0.85,
duration=1800.5,
search_snippets=["snippet 1", "snippet 2"],
@@ -443,7 +443,7 @@ class TestSearchResultModel:
assert result.title == "Test Title"
assert result.user_id == "user-123"
assert result.room_id == "room-456"
assert result.status == "completed"
assert result.status == "ended"
assert result.rank == 0.85
assert result.duration == 1800.5
assert len(result.search_snippets) == 2
@@ -474,7 +474,7 @@ class TestSearchResultModel:
id="test-id",
source_kind=SourceKind.LIVE,
created_at=datetime(2024, 6, 15, 12, 30, 45, tzinfo=timezone.utc),
status="completed",
status="ended",
rank=0.9,
duration=None,
search_snippets=[],

4
server/tests/test_search_long_summary.py
View File

@@ -25,7 +25,7 @@ async def test_long_summary_snippet_prioritization():
"id": test_id,
"name": "Test Snippet Priority",
"title": "Meeting About Projects",
"status": "completed",
"status": "ended",
"locked": False,
"duration": 1800.0,
"created_at": datetime.now(timezone.utc),
@@ -106,7 +106,7 @@ async def test_long_summary_only_search():
"id": test_id,
"name": "Test Long Only",
"title": "Standard Meeting",
"status": "completed",
"status": "ended",
"locked": False,
"duration": 1800.0,
"created_at": datetime.now(timezone.utc),

34
www/.env.example Normal file
View File

@@ -0,0 +1,34 @@
# Environment
ENVIRONMENT=development
NEXT_PUBLIC_ENV=development
# Site Configuration
NEXT_PUBLIC_SITE_URL=http://localhost:3000
# Nextauth envs
# not used in app code but in lib code
NEXTAUTH_URL=http://localhost:3000
NEXTAUTH_SECRET=your-nextauth-secret-here
# / Nextauth envs
# Authentication (Authentik OAuth/OIDC)
AUTHENTIK_ISSUER=https://authentik.example.com/application/o/reflector
AUTHENTIK_REFRESH_TOKEN_URL=https://authentik.example.com/application/o/token/
AUTHENTIK_CLIENT_ID=your-client-id-here
AUTHENTIK_CLIENT_SECRET=your-client-secret-here
# Feature Flags
# NEXT_PUBLIC_FEATURE_REQUIRE_LOGIN=true
# NEXT_PUBLIC_FEATURE_PRIVACY=false
# NEXT_PUBLIC_FEATURE_BROWSE=true
# NEXT_PUBLIC_FEATURE_SEND_TO_ZULIP=true
# NEXT_PUBLIC_FEATURE_ROOMS=true
# API URLs
NEXT_PUBLIC_API_URL=http://127.0.0.1:1250
NEXT_PUBLIC_WEBSOCKET_URL=ws://127.0.0.1:1250
NEXT_PUBLIC_AUTH_CALLBACK_URL=http://localhost:3000/auth-callback
# Sentry
# SENTRY_DSN=https://your-dsn@sentry.io/project-id
# SENTRY_IGNORE_API_RESOLUTION_ERROR=1

1
www/.gitignore vendored
View File

@@ -40,7 +40,6 @@ next-env.d.ts
# Sentry Auth Token
.sentryclirc
config.ts
# openapi logs
openapi-ts-error-*.log

30
www/app/(app)/AuthWrapper.tsx Normal file
View File

@@ -0,0 +1,30 @@
"use client";
import { Flex, Spinner } from "@chakra-ui/react";
import { useAuth } from "../lib/AuthProvider";
import { useLoginRequiredPages } from "../lib/useLoginRequiredPages";
export default function AuthWrapper({
children,
}: {
children: React.ReactNode;
}) {
const auth = useAuth();
const redirectPath = useLoginRequiredPages();
const redirectHappens = !!redirectPath;
if (auth.status === "loading" || redirectHappens) {
return (
<Flex
flexDir="column"
alignItems="center"
justifyContent="center"
h="calc(100vh - 80px)" // Account for header height
>
<Spinner size="xl" color="blue.500" />
</Flex>
);
}
return <>{children}</>;
}

7
www/app/(app)/browse/_components/FilterSidebar.tsx
View File

@@ -1,7 +1,10 @@
import React from "react";
import { Box, Stack, Link, Heading } from "@chakra-ui/react";
import NextLink from "next/link";
import { Room, SourceKind } from "../../../api";
import type { components } from "../../../reflector-api";
type Room = components["schemas"]["Room"];
type SourceKind = components["schemas"]["SourceKind"];
interface FilterSidebarProps {
rooms: Room[];
@@ -72,7 +75,7 @@ export default function FilterSidebar({
key={room.id}
as={NextLink}
href="#"
onClick={() => onFilterChange("room", room.id)}
onClick={() => onFilterChange("room" as SourceKind, room.id)}
color={
selectedSourceKind === "room" && selectedRoomId === room.id
? "blue.500"

7
www/app/(app)/browse/_components/TranscriptCards.tsx
View File

@@ -18,7 +18,10 @@ import {
highlightMatches,
generateTextFragment,
} from "../../../lib/textHighlight";
import { SearchResult } from "../../../api";
import type { components } from "../../../reflector-api";
type SearchResult = components["schemas"]["SearchResult"];
type SourceKind = components["schemas"]["SourceKind"];
interface TranscriptCardsProps {
results: SearchResult[];
@@ -120,7 +123,7 @@ function TranscriptCard({
: "N/A";
const formattedDate = formatLocalDate(result.created_at);
const source =
result.source_kind === "room"
result.source_kind === ("room" as SourceKind)
? result.room_name || result.room_id
: result.source_kind;

3
www/app/(app)/browse/_components/TranscriptStatusIcon.tsx
View File

@@ -7,9 +7,10 @@ import {
FaMicrophone,
FaGear,
} from "react-icons/fa6";
import { TranscriptStatus } from "../../../lib/transcript";
interface TranscriptStatusIconProps {
status: string;
status: TranscriptStatus;
}
export default function TranscriptStatusIcon({

149
www/app/(app)/browse/page.tsx
View File

@@ -19,37 +19,33 @@ import {
parseAsStringLiteral,
} from "nuqs";
import { LuX } from "react-icons/lu";
import { useSearchTranscripts } from "../transcripts/useSearchTranscripts";
import useSessionUser from "../../lib/useSessionUser";
import { Room, SourceKind, SearchResult, $SourceKind } from "../../api";
import useApi from "../../lib/useApi";
import { useError } from "../../(errors)/errorContext";
import type { components } from "../../reflector-api";
type Room = components["schemas"]["Room"];
type SourceKind = components["schemas"]["SourceKind"];
type SearchResult = components["schemas"]["SearchResult"];
import {
useRoomsList,
useTranscriptsSearch,
useTranscriptDelete,
useTranscriptProcess,
} from "../../lib/apiHooks";
import FilterSidebar from "./_components/FilterSidebar";
import Pagination, {
FIRST_PAGE,
PaginationPage,
parsePaginationPage,
totalPages as getTotalPages,
paginationPageTo0Based,
} from "./_components/Pagination";
import TranscriptCards from "./_components/TranscriptCards";
import DeleteTranscriptDialog from "./_components/DeleteTranscriptDialog";
import { formatLocalDate } from "../../lib/time";
import { RECORD_A_MEETING_URL } from "../../api/urls";
import { useUserName } from "../../lib/useUserName";
const SEARCH_FORM_QUERY_INPUT_NAME = "query" as const;
const usePrefetchRooms = (setRooms: (rooms: Room[]) => void): void => {
const { setError } = useError();
const api = useApi();
useEffect(() => {
if (!api) return;
api
.v1RoomsList({ page: 1 })
.then((rooms) => setRooms(rooms.items))
.catch((err) => setError(err, "There was an error fetching the rooms"));
}, [api, setError]);
};
const SearchForm: React.FC<{
setPage: (page: PaginationPage) => void;
sourceKind: SourceKind | null;
@@ -69,7 +65,6 @@ const SearchForm: React.FC<{
searchQuery,
setSearchQuery,
}) => {
// to keep the search input controllable + more fine grained control (urlSearchQuery is updated on submits)
const [searchInputValue, setSearchInputValue] = useState(searchQuery || "");
const handleSearchQuerySubmit = async (d: FormData) => {
await setSearchQuery((d.get(SEARCH_FORM_QUERY_INPUT_NAME) as string) || "");
@@ -163,7 +158,6 @@ const UnderSearchFormFilterIndicators: React.FC<{
p="1px"
onClick={() => {
setSourceKind(null);
// TODO questionable
setRoomId(null);
}}
_hover={{ bg: "blue.200" }}
@@ -209,7 +203,11 @@ export default function TranscriptBrowser() {
const [urlSourceKind, setUrlSourceKind] = useQueryState(
"source",
parseAsStringLiteral($SourceKind.enum).withOptions({
parseAsStringLiteral([
"room",
"live",
"file",
] as const satisfies SourceKind[]).withOptions({
shallow: false,
}),
);
@@ -229,46 +227,40 @@ export default function TranscriptBrowser() {
useEffect(() => {
const maybePage = parsePaginationPage(urlPage);
if ("error" in maybePage) {
setPage(FIRST_PAGE).then(() => {
/*may be called n times we dont care*/
});
setPage(FIRST_PAGE).then(() => {});
return;
}
_setSafePage(maybePage.value);
}, [urlPage]);
const [rooms, setRooms] = useState<Room[]>([]);
const pageSize = 20;
const {
results,
totalCount: totalResults,
isLoading,
reload,
} = useSearchTranscripts(
urlSearchQuery,
{
roomIds: urlRoomId ? [urlRoomId] : null,
sourceKind: urlSourceKind,
},
{
pageSize,
page,
},
);
data: searchData,
isLoading: searchLoading,
refetch: reloadSearch,
} = useTranscriptsSearch(urlSearchQuery, {
limit: pageSize,
offset: paginationPageTo0Based(page) * pageSize,
room_id: urlRoomId || undefined,
source_kind: urlSourceKind || undefined,
});
const results = searchData?.results || [];
const totalResults = searchData?.total || 0;
// Fetch rooms
const { data: roomsData } = useRoomsList(1);
const rooms = roomsData?.items || [];
const totalPages = getTotalPages(totalResults, pageSize);
const userName = useSessionUser().name;
const userName = useUserName();
const [deletionLoading, setDeletionLoading] = useState(false);
const api = useApi();
const { setError } = useError();
const cancelRef = React.useRef(null);
const [transcriptToDeleteId, setTranscriptToDeleteId] =
React.useState<string>();
usePrefetchRooms(setRooms);
const handleFilterTranscripts = (
sourceKind: SourceKind | null,
roomId: string,
@@ -280,44 +272,37 @@ export default function TranscriptBrowser() {
const onCloseDeletion = () => setTranscriptToDeleteId(undefined);
const deleteTranscript = useTranscriptDelete();
const processTranscript = useTranscriptProcess();
const confirmDeleteTranscript = (transcriptId: string) => {
if (!api || deletionLoading) return;
if (deletionLoading) return;
setDeletionLoading(true);
api
.v1TranscriptDelete({ transcriptId })
.then(() => {
setDeletionLoading(false);
onCloseDeletion();
reload();
})
.catch((err) => {
setDeletionLoading(false);
setError(err, "There was an error deleting the transcript");
});
deleteTranscript.mutate(
{
params: {
path: { transcript_id: transcriptId },
},
},
{
onSuccess: () => {
setDeletionLoading(false);
onCloseDeletion();
reloadSearch();
},
onError: () => {
setDeletionLoading(false);
},
},
);
};
const handleProcessTranscript = (transcriptId: string) => {
if (!api) {
console.error("API not available on handleProcessTranscript");
return;
}
api
.v1TranscriptProcess({ transcriptId })
.then((result) => {
const status =
result && typeof result === "object" && "status" in result
? (result as { status: string }).status
: undefined;
if (status === "already running") {
setError(
new Error("Processing is already running, please wait"),
"Processing is already running, please wait",
);
}
})
.catch((err) => {
setError(err, "There was an error processing the transcript");
});
processTranscript.mutate({
params: {
path: { transcript_id: transcriptId },
},
});
};
const transcriptToDelete = results?.find(
@@ -332,7 +317,7 @@ export default function TranscriptBrowser() {
? transcriptToDelete.room_name || transcriptToDelete.room_id
: transcriptToDelete?.source_kind;
if (isLoading && results.length === 0) {
if (searchLoading && results.length === 0) {
return (
<Flex
flexDir="column"
@@ -360,7 +345,7 @@ export default function TranscriptBrowser() {
>
<Heading size="lg">
{userName ? `${userName}'s Transcriptions` : "Your Transcriptions"}{" "}
{(isLoading || deletionLoading) && <Spinner size="sm" />}
{(searchLoading || deletionLoading) && <Spinner size="sm" />}
</Heading>
</Flex>
@@ -403,12 +388,12 @@ export default function TranscriptBrowser() {
<TranscriptCards
results={results}
query={urlSearchQuery}
isLoading={isLoading}
isLoading={searchLoading}
onDelete={setTranscriptToDeleteId}
onReprocess={handleProcessTranscript}
/>
{!isLoading && results.length === 0 && (
{!searchLoading && results.length === 0 && (
<EmptyResult searchQuery={urlSearchQuery} />
)}
</Flex>

15
www/app/(app)/layout.tsx
View File

@@ -1,10 +1,9 @@
import { Container, Flex, Link } from "@chakra-ui/react";
import { getConfig } from "../lib/edgeConfig";
import { featureEnabled } from "../lib/features";
import NextLink from "next/link";
import Image from "next/image";
import About from "../(aboutAndPrivacy)/about";
import Privacy from "../(aboutAndPrivacy)/privacy";
import UserInfo from "../(auth)/userInfo";
import AuthWrapper from "./AuthWrapper";
import { RECORD_A_MEETING_URL } from "../api/urls";
export default async function AppLayout({
@@ -12,8 +11,6 @@ export default async function AppLayout({
}: {
children: React.ReactNode;
}) {
const config = await getConfig();
const { requireLogin, privacy, browse, rooms } = config.features;
return (
<Container
minW="100vw"
@@ -59,7 +56,7 @@ export default async function AppLayout({
>
Create
</Link>
{browse ? (
{featureEnabled("browse") ? (
<>
&nbsp;·&nbsp;
<Link href="/browse" as={NextLink} className="font-light px-2">
@@ -69,7 +66,7 @@ export default async function AppLayout({
) : (
<></>
)}
{rooms ? (
{featureEnabled("rooms") ? (
<>
&nbsp;·&nbsp;
<Link href="/rooms" as={NextLink} className="font-light px-2">
@@ -79,7 +76,7 @@ export default async function AppLayout({
) : (
<></>
)}
{requireLogin ? (
{featureEnabled("requireLogin") ? (
<>
&nbsp;·&nbsp;
<UserInfo />
@@ -90,7 +87,7 @@ export default async function AppLayout({
</div>
</Flex>
{children}
<AuthWrapper>{children}</AuthWrapper>
</Container>
);
}

6
www/app/(app)/rooms/_components/RoomCards.tsx
View File

@@ -13,7 +13,9 @@ import {
Badge,
} from "@chakra-ui/react";
import { LuLink } from "react-icons/lu";
import { RoomDetails } from "../../../api";
import type { components } from "../../../reflector-api";
type Room = components["schemas"]["Room"];
import { RoomActionsMenu } from "./RoomActionsMenu";
import {
getPlatformDisplayName,
@@ -21,7 +23,7 @@ import {
} from "../../../lib/videoPlatforms";
interface RoomCardsProps {
rooms: RoomDetails[];
rooms: Room[];
linkCopied: string;
onCopyUrl: (roomName: string) => void;
onEdit: (roomId: string, roomData: any) => void;

6
www/app/(app)/rooms/_components/RoomList.tsx
View File

@@ -1,11 +1,13 @@
import { Box, Heading, Text, VStack } from "@chakra-ui/react";
import { RoomDetails } from "../../../api";
import type { components } from "../../../reflector-api";
type Room = components["schemas"]["Room"];
import { RoomTable } from "./RoomTable";
import { RoomCards } from "./RoomCards";
interface RoomListProps {
title: string;
rooms: RoomDetails[];
rooms: Room[];
linkCopied: string;
onCopyUrl: (roomName: string) => void;
onEdit: (roomId: string, roomData: any) => void;

6
www/app/(app)/rooms/_components/RoomTable.tsx
View File

@@ -10,7 +10,9 @@ import {
Badge,
} from "@chakra-ui/react";
import { LuLink } from "react-icons/lu";
import { RoomDetails } from "../../../api";
import type { components } from "../../../reflector-api";
type Room = components["schemas"]["Room"];
import { RoomActionsMenu } from "./RoomActionsMenu";
import {
getPlatformDisplayName,
@@ -18,7 +20,7 @@ import {
} from "../../../lib/videoPlatforms";
interface RoomTableProps {
rooms: RoomDetails[];
rooms: Room[];
linkCopied: string;
onCopyUrl: (roomName: string) => void;
onEdit: (roomId: string, roomData: any) => void;

233
www/app/(app)/rooms/page.tsx
View File

@@ -15,13 +15,24 @@ import {
createListCollection,
useDisclosure,
} from "@chakra-ui/react";
import { useEffect, useState } from "react";
import { useEffect, useMemo, useState } from "react";
import { LuEye, LuEyeOff } from "react-icons/lu";
import useApi from "../../lib/useApi";
import useRoomList from "./useRoomList";
import { ApiError, RoomDetails } from "../../api";
import type { components } from "../../reflector-api";
import {
useRoomCreate,
useRoomUpdate,
useRoomDelete,
useZulipStreams,
useZulipTopics,
useRoomGet,
useRoomTestWebhook,
} from "../../lib/apiHooks";
import { RoomList } from "./_components/RoomList";
import { PaginationPage } from "../browse/_components/Pagination";
import { assertExists } from "../../lib/utils";
type Room = components["schemas"]["Room"];
interface SelectOption {
label: string;
@@ -76,66 +87,77 @@ export default function RoomsList() {
const recordingTypeCollection = createListCollection({
items: recordingTypeOptions,
});
const [room, setRoom] = useState(roomInitialState);
const [roomInput, setRoomInput] = useState<null | typeof roomInitialState>(
null,
);
const [isEditing, setIsEditing] = useState(false);
const [editRoomId, setEditRoomId] = useState("");
const api = useApi();
// TODO seems to be no setPage calls
const [page, setPage] = useState<number>(1);
const { loading, response, refetch } = useRoomList(PaginationPage(page));
const [streams, setStreams] = useState<Stream[]>([]);
const [topics, setTopics] = useState<Topic[]>([]);
const [editRoomId, setEditRoomId] = useState<string | null>(null);
const {
loading,
response,
refetch,
error: roomListError,
} = useRoomList(PaginationPage(1));
const [nameError, setNameError] = useState("");
const [linkCopied, setLinkCopied] = useState("");
const [selectedStreamId, setSelectedStreamId] = useState<number | null>(null);
const [testingWebhook, setTestingWebhook] = useState(false);
const [webhookTestResult, setWebhookTestResult] = useState<string | null>(
null,
);
const [showWebhookSecret, setShowWebhookSecret] = useState(false);
interface Stream {
stream_id: number;
name: string;
}
interface Topic {
name: string;
}
const createRoomMutation = useRoomCreate();
const updateRoomMutation = useRoomUpdate();
const deleteRoomMutation = useRoomDelete();
const { data: streams = [] } = useZulipStreams();
const { data: topics = [] } = useZulipTopics(selectedStreamId);
const {
data: detailedEditedRoom,
isLoading: isDetailedEditedRoomLoading,
error: detailedEditedRoomError,
} = useRoomGet(editRoomId);
const error = roomListError || detailedEditedRoomError;
// room being edited, as fetched from the server
const editedRoom: typeof roomInitialState | null = useMemo(
() =>
detailedEditedRoom
? {
name: detailedEditedRoom.name,
zulipAutoPost: detailedEditedRoom.zulip_auto_post,
zulipStream: detailedEditedRoom.zulip_stream,
zulipTopic: detailedEditedRoom.zulip_topic,
isLocked: detailedEditedRoom.is_locked,
roomMode: detailedEditedRoom.room_mode,
recordingType: detailedEditedRoom.recording_type,
recordingTrigger: detailedEditedRoom.recording_trigger,
isShared: detailedEditedRoom.is_shared,
webhookUrl: detailedEditedRoom.webhook_url || "",
webhookSecret: detailedEditedRoom.webhook_secret || "",
}
: null,
[detailedEditedRoom],
);
// a room input value or a last api room state
const room = roomInput || editedRoom || roomInitialState;
const roomTestWebhookMutation = useRoomTestWebhook();
// Update selected stream ID when zulip stream changes
useEffect(() => {
const fetchZulipStreams = async () => {
if (!api) return;
try {
const response = await api.v1ZulipGetStreams();
setStreams(response);
} catch (error) {
console.error("Error fetching Zulip streams:", error);
if (room.zulipStream && streams.length > 0) {
const selectedStream = streams.find((s) => s.name === room.zulipStream);
if (selectedStream !== undefined) {
setSelectedStreamId(selectedStream.stream_id);
}
};
if (room.zulipAutoPost) {
fetchZulipStreams();
} else {
setSelectedStreamId(null);
}
}, [room.zulipAutoPost, !api]);
useEffect(() => {
const fetchZulipTopics = async () => {
if (!api || !room.zulipStream) return;
try {
const selectedStream = streams.find((s) => s.name === room.zulipStream);
if (selectedStream) {
const response = await api.v1ZulipGetTopics({
streamId: selectedStream.stream_id,
});
setTopics(response);
}
} catch (error) {
console.error("Error fetching Zulip topics:", error);
}
};
fetchZulipTopics();
}, [room.zulipStream, streams, api]);
}, [room.zulipStream, streams]);
const streamOptions: SelectOption[] = streams.map((stream) => {
return { label: stream.name, value: stream.name };
@@ -167,35 +189,42 @@ export default function RoomsList() {
const handleCloseDialog = () => {
setShowWebhookSecret(false);
setWebhookTestResult(null);
setEditRoomId(null);
onClose();
};
const handleTestWebhook = async () => {
if (!room.webhookUrl || !editRoomId) {
if (!room.webhookUrl) {
setWebhookTestResult("Please enter a webhook URL first");
return;
}
if (!editRoomId) {
console.error("No room ID to test webhook");
return;
}
setTestingWebhook(true);
setWebhookTestResult(null);
try {
const response = await api?.v1RoomsTestWebhook({
roomId: editRoomId,
const response = await roomTestWebhookMutation.mutateAsync({
params: {
path: {
room_id: editRoomId,
},
},
});
if (response?.success) {
if (response.success) {
setWebhookTestResult(
`✅ Webhook test successful! Status: ${response.status_code}`,
);
} else {
let errorMsg = `❌ Webhook test failed`;
if (response?.status_code) {
errorMsg += ` (Status: ${response.status_code})`;
}
if (response?.error) {
errorMsg += ` (Status: ${response.status_code})`;
if (response.error) {
errorMsg += `: ${response.error}`;
} else if (response?.response_preview) {
} else if (response.response_preview) {
// Try to parse and extract meaningful error from response
// Specific to N8N at the moment, as there is no specification for that
// We could just display as is, but decided here to dig a little bit more.
@@ -249,27 +278,29 @@ export default function RoomsList() {
};
if (isEditing) {
await api?.v1RoomsUpdate({
roomId: editRoomId,
requestBody: roomData,
await updateRoomMutation.mutateAsync({
params: {
path: { room_id: assertExists(editRoomId) },
},
body: roomData,
});
} else {
await api?.v1RoomsCreate({
requestBody: roomData,
await createRoomMutation.mutateAsync({
body: roomData,
});
}
setRoom(roomInitialState);
setRoomInput(null);
setIsEditing(false);
setEditRoomId("");
setNameError("");
refetch();
onClose();
handleCloseDialog();
} catch (err) {
} catch (err: any) {
if (
err instanceof ApiError &&
err.status === 400 &&
(err.body as any).detail == "Room name is not unique"
err?.status === 400 &&
err?.body?.detail == "Room name is not unique"
) {
setNameError(
"This room name is already taken. Please choose a different name.",
@@ -280,46 +311,11 @@ export default function RoomsList() {
}
};
const handleEditRoom = async (roomId, roomData) => {
const handleEditRoom = async (roomId: string, roomData) => {
// Reset states
setShowWebhookSecret(false);
setWebhookTestResult(null);
// Fetch full room details to get webhook fields
try {
const detailedRoom = await api?.v1RoomsGet({ roomId });
if (detailedRoom) {
setRoom({
name: detailedRoom.name,
zulipAutoPost: detailedRoom.zulip_auto_post,
zulipStream: detailedRoom.zulip_stream,
zulipTopic: detailedRoom.zulip_topic,
isLocked: detailedRoom.is_locked,
roomMode: detailedRoom.room_mode,
recordingType: detailedRoom.recording_type,
recordingTrigger: detailedRoom.recording_trigger,
isShared: detailedRoom.is_shared,
webhookUrl: detailedRoom.webhook_url || "",
webhookSecret: detailedRoom.webhook_secret || "",
});
}
} catch (error) {
console.error("Failed to fetch room details, using list data:", error);
// Fallback to using the data from the list
setRoom({
name: roomData.name,
zulipAutoPost: roomData.zulip_auto_post,
zulipStream: roomData.zulip_stream,
zulipTopic: roomData.zulip_topic,
isLocked: roomData.is_locked,
roomMode: roomData.room_mode,
recordingType: roomData.recording_type,
recordingTrigger: roomData.recording_trigger,
isShared: roomData.is_shared,
webhookUrl: roomData.webhook_url || "",
webhookSecret: roomData.webhook_secret || "",
});
}
setEditRoomId(roomId);
setIsEditing(true);
setNameError("");
@@ -328,8 +324,10 @@ export default function RoomsList() {
const handleDeleteRoom = async (roomId: string) => {
try {
await api?.v1RoomsDelete({
roomId,
await deleteRoomMutation.mutateAsync({
params: {
path: { room_id: roomId },
},
});
refetch();
} catch (err) {
@@ -346,15 +344,15 @@ export default function RoomsList() {
.toLowerCase();
setNameError("");
}
setRoom({
setRoomInput({
...room,
[name]: type === "checkbox" ? checked : value,
});
};
const myRooms: RoomDetails[] =
const myRooms: Room[] =
response?.items.filter((roomData) => !roomData.is_shared) || [];
const sharedRooms: RoomDetails[] =
const sharedRooms: Room[] =
response?.items.filter((roomData) => roomData.is_shared) || [];
if (loading && !response)
@@ -369,6 +367,9 @@ export default function RoomsList() {
</Flex>
);
if (roomListError)
return <div>{`${roomListError.name}: ${roomListError.message}`}</div>;
return (
<Flex
flexDir="column"
@@ -387,7 +388,7 @@ export default function RoomsList() {
colorPalette="primary"
onClick={() => {
setIsEditing(false);
setRoom(roomInitialState);
setRoomInput(null);
setNameError("");
setShowWebhookSecret(false);
setWebhookTestResult(null);
@@ -456,7 +457,7 @@ export default function RoomsList() {
<Select.Root
value={[room.roomMode]}
onValueChange={(e) =>
setRoom({ ...room, roomMode: e.value[0] })
setRoomInput({ ...room, roomMode: e.value[0] })
}
collection={roomModeCollection}
>
@@ -486,7 +487,7 @@ export default function RoomsList() {
<Select.Root
value={[room.recordingType]}
onValueChange={(e) =>
setRoom({
setRoomInput({
...room,
recordingType: e.value[0],
recordingTrigger:
@@ -521,7 +522,7 @@ export default function RoomsList() {
<Select.Root
value={[room.recordingTrigger]}
onValueChange={(e) =>
setRoom({ ...room, recordingTrigger: e.value[0] })
setRoomInput({ ...room, recordingTrigger: e.value[0] })
}
collection={recordingTriggerCollection}
disabled={room.recordingType !== "cloud"}
@@ -576,7 +577,7 @@ export default function RoomsList() {
<Select.Root
value={room.zulipStream ? [room.zulipStream] : []}
onValueChange={(e) =>
setRoom({
setRoomInput({
...room,
zulipStream: e.value[0],
zulipTopic: "",
@@ -611,7 +612,7 @@ export default function RoomsList() {
<Select.Root
value={room.zulipTopic ? [room.zulipTopic] : []}
onValueChange={(e) =>
setRoom({ ...room, zulipTopic: e.value[0] })
setRoomInput({ ...room, zulipTopic: e.value[0] })
}
collection={topicCollection}
disabled={!room.zulipAutoPost}

49
www/app/(app)/rooms/useRoomList.tsx
View File

@@ -1,48 +1,27 @@
import { useEffect, useState } from "react";
import { useError } from "../../(errors)/errorContext";
import useApi from "../../lib/useApi";
import { Page_RoomDetails_ } from "../../api";
import { useRoomsList } from "../../lib/apiHooks";
import type { components } from "../../reflector-api";
type Page_Room_ = components["schemas"]["Page_RoomDetails_"];
import { PaginationPage } from "../browse/_components/Pagination";
type RoomList = {
response: Page_RoomDetails_ | null;
response: Page_Room_ | null;
loading: boolean;
error: Error | null;
refetch: () => void;
};
//always protected
// Wrapper to maintain backward compatibility
const useRoomList = (page: PaginationPage): RoomList => {
const [response, setResponse] = useState<Page_RoomDetails_ | null>(null);
const [loading, setLoading] = useState<boolean>(true);
const [error, setErrorState] = useState<Error | null>(null);
const { setError } = useError();
const api = useApi();
const [refetchCount, setRefetchCount] = useState(0);
const refetch = () => {
setLoading(true);
setRefetchCount(refetchCount + 1);
const { data, isLoading, error, refetch } = useRoomsList(page);
return {
response: data || null,
loading: isLoading,
error: error
? new Error(error.detail ? JSON.stringify(error.detail) : undefined)
: null,
refetch,
};
useEffect(() => {
if (!api) return;
setLoading(true);
api
.v1RoomsList({ page })
.then((response) => {
setResponse(response);
setLoading(false);
})
.catch((err) => {
setResponse(null);
setLoading(false);
setError(err);
setErrorState(err);
});
}, [!api, page, refetchCount]);
return { response, loading, error, refetch };
};
export default useRoomList;

6
www/app/(app)/transcripts/[transcriptId]/_components/TopicList.tsx
View File

@@ -3,8 +3,10 @@ import ScrollToBottom from "../../scrollToBottom";
import { Topic } from "../../webSocketTypes";
import useParticipants from "../../useParticipants";
import { Box, Flex, Text, Accordion } from "@chakra-ui/react";
import { featureEnabled } from "../../../../domainContext";
import { TopicItem } from "./TopicItem";
import { TranscriptStatus } from "../../../../lib/transcript";
import { featureEnabled } from "../../../../lib/features";
type TopicListProps = {
topics: Topic[];
@@ -14,7 +16,7 @@ type TopicListProps = {
];
autoscroll: boolean;
transcriptId: string;
status: string;
status: TranscriptStatus | null;
currentTranscriptText: any;
};

52
www/app/(app)/transcripts/[transcriptId]/correct/page.tsx
View File

@@ -1,30 +1,35 @@
"use client";
import { useState } from "react";
import { useState, use } from "react";
import TopicHeader from "./topicHeader";
import TopicWords from "./topicWords";
import TopicPlayer from "./topicPlayer";
import useParticipants from "../../useParticipants";
import useTopicWithWords from "../../useTopicWithWords";
import ParticipantList from "./participantList";
import { GetTranscriptTopic } from "../../../../api";
import type { components } from "../../../../reflector-api";
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
import { SelectedText, selectedTextIsTimeSlice } from "./types";
import useApi from "../../../../lib/useApi";
import useTranscript from "../../useTranscript";
import {
useTranscriptGet,
useTranscriptUpdate,
} from "../../../../lib/apiHooks";
import { useError } from "../../../../(errors)/errorContext";
import { useRouter } from "next/navigation";
import { Box, Grid } from "@chakra-ui/react";
export type TranscriptCorrect = {
params: {
params: Promise<{
transcriptId: string;
};
}>;
};
export default function TranscriptCorrect({
params: { transcriptId },
}: TranscriptCorrect) {
const api = useApi();
const transcript = useTranscript(transcriptId);
export default function TranscriptCorrect(props: TranscriptCorrect) {
const params = use(props.params);
const { transcriptId } = params;
const updateTranscriptMutation = useTranscriptUpdate();
const transcript = useTranscriptGet(transcriptId);
const stateCurrentTopic = useState<GetTranscriptTopic>();
const [currentTopic, _sct] = stateCurrentTopic;
const stateSelectedText = useState<SelectedText>();
@@ -34,16 +39,21 @@ export default function TranscriptCorrect({
const { setError } = useError();
const router = useRouter();
const markAsDone = () => {
if (transcript.response && !transcript.response.reviewed) {
api
?.v1TranscriptUpdate({ transcriptId, requestBody: { reviewed: true } })
.then(() => {
router.push(`/transcripts/${transcriptId}`);
})
.catch((e) => {
setError(e, "Error marking as done");
const markAsDone = async () => {
if (transcript.data && !transcript.data.reviewed) {
try {
await updateTranscriptMutation.mutateAsync({
params: {
path: {
transcript_id: transcriptId,
},
},
body: { reviewed: true },
});
router.push(`/transcripts/${transcriptId}`);
} catch (e) {
setError(e as Error, "Error marking as done");
}
}
};
@@ -108,7 +118,7 @@ export default function TranscriptCorrect({
}}
/>
</Grid>
{transcript.response && !transcript.response?.reviewed && (
{transcript.data && !transcript.data?.reviewed && (
<div className="flex flex-row justify-end">
<button
className="p-2 px-4 rounded bg-green-400"

197
www/app/(app)/transcripts/[transcriptId]/correct/participantList.tsx
View File

@@ -1,8 +1,15 @@
import { faArrowTurnDown } from "@fortawesome/free-solid-svg-icons";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { ChangeEvent, useEffect, useRef, useState } from "react";
import { Participant } from "../../../../api";
import useApi from "../../../../lib/useApi";
import type { components } from "../../../../reflector-api";
type Participant = components["schemas"]["Participant"];
import {
useTranscriptSpeakerAssign,
useTranscriptSpeakerMerge,
useTranscriptParticipantUpdate,
useTranscriptParticipantCreate,
useTranscriptParticipantDelete,
} from "../../../../lib/apiHooks";
import { UseParticipants } from "../../useParticipants";
import { selectedTextIsSpeaker, selectedTextIsTimeSlice } from "./types";
import { useError } from "../../../../(errors)/errorContext";
@@ -30,9 +37,19 @@ const ParticipantList = ({
topicWithWords,
stateSelectedText,
}: ParticipantList) => {
const api = useApi();
const { setError } = useError();
const [loading, setLoading] = useState(false);
const speakerAssignMutation = useTranscriptSpeakerAssign();
const speakerMergeMutation = useTranscriptSpeakerMerge();
const participantUpdateMutation = useTranscriptParticipantUpdate();
const participantCreateMutation = useTranscriptParticipantCreate();
const participantDeleteMutation = useTranscriptParticipantDelete();
const loading =
speakerAssignMutation.isPending ||
speakerMergeMutation.isPending ||
participantUpdateMutation.isPending ||
participantCreateMutation.isPending ||
participantDeleteMutation.isPending;
const [participantInput, setParticipantInput] = useState("");
const inputRef = useRef<HTMLInputElement>(null);
const [selectedText, setSelectedText] = stateSelectedText;
@@ -103,7 +120,6 @@ const ParticipantList = ({
const onSuccess = () => {
topicWithWords.refetch();
participants.refetch();
setLoading(false);
setAction(null);
setSelectedText(undefined);
setSelectedParticipant(undefined);
@@ -120,11 +136,14 @@ const ParticipantList = ({
if (loading || participants.loading || topicWithWords.loading) return;
if (!selectedTextIsTimeSlice(selectedText)) return;
setLoading(true);
try {
await api?.v1TranscriptAssignSpeaker({
transcriptId,
requestBody: {
await speakerAssignMutation.mutateAsync({
params: {
path: {
transcript_id: transcriptId,
},
},
body: {
participant: participant.id,
timestamp_from: selectedText.start,
timestamp_to: selectedText.end,
@@ -132,8 +151,7 @@ const ParticipantList = ({
});
onSuccess();
} catch (error) {
setError(error, "There was an error assigning");
setLoading(false);
setError(error as Error, "There was an error assigning");
throw error;
}
};
@@ -141,32 +159,38 @@ const ParticipantList = ({
const mergeSpeaker =
(speakerFrom, participantTo: Participant) => async () => {
if (loading || participants.loading || topicWithWords.loading) return;
setLoading(true);
if (participantTo.speaker) {
try {
await api?.v1TranscriptMergeSpeaker({
transcriptId,
requestBody: {
await speakerMergeMutation.mutateAsync({
params: {
path: {
transcript_id: transcriptId,
},
},
body: {
speaker_from: speakerFrom,
speaker_to: participantTo.speaker,
},
});
onSuccess();
} catch (error) {
setError(error, "There was an error merging");
setLoading(false);
setError(error as Error, "There was an error merging");
}
} else {
try {
await api?.v1TranscriptUpdateParticipant({
transcriptId,
participantId: participantTo.id,
requestBody: { speaker: speakerFrom },
await participantUpdateMutation.mutateAsync({
params: {
path: {
transcript_id: transcriptId,
participant_id: participantTo.id,
},
},
body: { speaker: speakerFrom },
});
onSuccess();
} catch (error) {
setError(error, "There was an error merging (update)");
setLoading(false);
setError(error as Error, "There was an error merging (update)");
}
}
};
@@ -186,105 +210,106 @@ const ParticipantList = ({
(p) => p.speaker == selectedText,
);
if (participant && participant.name !== participantInput) {
setLoading(true);
api
?.v1TranscriptUpdateParticipant({
transcriptId,
participantId: participant.id,
requestBody: {
try {
await participantUpdateMutation.mutateAsync({
params: {
path: {
transcript_id: transcriptId,
participant_id: participant.id,
},
},
body: {
name: participantInput,
},
})
.then(() => {
participants.refetch();
setLoading(false);
setAction(null);
})
.catch((e) => {
setError(e, "There was an error renaming");
setLoading(false);
});
participants.refetch();
setAction(null);
} catch (e) {
setError(e as Error, "There was an error renaming");
}
}
} else if (
action == "Create to rename" &&
selectedTextIsSpeaker(selectedText)
) {
setLoading(true);
api
?.v1TranscriptAddParticipant({
transcriptId,
requestBody: {
try {
await participantCreateMutation.mutateAsync({
params: {
path: {
transcript_id: transcriptId,
},
},
body: {
name: participantInput,
speaker: selectedText,
},
})
.then(() => {
participants.refetch();
setParticipantInput("");
setOneMatch(undefined);
setLoading(false);
})
.catch((e) => {
setError(e, "There was an error creating");
setLoading(false);
});
participants.refetch();
setParticipantInput("");
setOneMatch(undefined);
} catch (e) {
setError(e as Error, "There was an error creating");
}
} else if (
action == "Create and assign" &&
selectedTextIsTimeSlice(selectedText)
) {
setLoading(true);
try {
const participant = await api?.v1TranscriptAddParticipant({
transcriptId,
requestBody: {
const participant = await participantCreateMutation.mutateAsync({
params: {
path: {
transcript_id: transcriptId,
},
},
body: {
name: participantInput,
},
});
setLoading(false);
assignTo(participant)().catch(() => {
// error and loading are handled by assignTo catch
participants.refetch();
});
} catch (error) {
setError(e, "There was an error creating");
setLoading(false);
setError(error as Error, "There was an error creating");
}
} else if (action == "Create") {
setLoading(true);
api
?.v1TranscriptAddParticipant({
transcriptId,
requestBody: {
try {
await participantCreateMutation.mutateAsync({
params: {
path: {
transcript_id: transcriptId,
},
},
body: {
name: participantInput,
},
})
.then(() => {
participants.refetch();
setParticipantInput("");
setLoading(false);
inputRef.current?.focus();
})
.catch((e) => {
setError(e, "There was an error creating");
setLoading(false);
});
participants.refetch();
setParticipantInput("");
inputRef.current?.focus();
} catch (e) {
setError(e as Error, "There was an error creating");
}
}
};
const deleteParticipant = (participantId) => (e) => {
const deleteParticipant = (participantId) => async (e) => {
e.stopPropagation();
if (loading || participants.loading || topicWithWords.loading) return;
setLoading(true);
api
?.v1TranscriptDeleteParticipant({ transcriptId, participantId })
.then(() => {
participants.refetch();
setLoading(false);
})
.catch((e) => {
setError(e, "There was an error deleting");
setLoading(false);
try {
await participantDeleteMutation.mutateAsync({
params: {
path: {
transcript_id: transcriptId,
participant_id: participantId,
},
},
});
participants.refetch();
} catch (e) {
setError(e as Error, "There was an error deleting");
}
};
const selectParticipant = (participant) => (e) => {

3
www/app/(app)/transcripts/[transcriptId]/correct/topicHeader.tsx
View File

@@ -1,6 +1,7 @@
import useTopics from "../../useTopics";
import { Dispatch, SetStateAction, useEffect } from "react";
import { GetTranscriptTopic } from "../../../../api";
import type { components } from "../../../../reflector-api";
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
import {
BoxProps,
Box,

37
www/app/(app)/transcripts/[transcriptId]/finalSummary.tsx
View File

@@ -2,12 +2,10 @@ import { useEffect, useRef, useState } from "react";
import React from "react";
import Markdown from "react-markdown";
import "../../../styles/markdown.css";
import {
GetTranscript,
GetTranscriptTopic,
UpdateTranscript,
} from "../../../api";
import useApi from "../../../lib/useApi";
import type { components } from "../../../reflector-api";
type GetTranscript = components["schemas"]["GetTranscript"];
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
import { useTranscriptUpdate } from "../../../lib/apiHooks";
import {
Flex,
Heading,
@@ -33,9 +31,8 @@ export default function FinalSummary(props: FinalSummaryProps) {
const [preEditSummary, setPreEditSummary] = useState("");
const [editedSummary, setEditedSummary] = useState("");
const api = useApi();
const { setError } = useError();
const updateTranscriptMutation = useTranscriptUpdate();
useEffect(() => {
setEditedSummary(props.transcriptResponse?.long_summary || "");
@@ -47,12 +44,15 @@ export default function FinalSummary(props: FinalSummaryProps) {
const updateSummary = async (newSummary: string, transcriptId: string) => {
try {
const requestBody: UpdateTranscript = {
long_summary: newSummary,
};
const updatedTranscript = await api?.v1TranscriptUpdate({
transcriptId,
requestBody,
const updatedTranscript = await updateTranscriptMutation.mutateAsync({
params: {
path: {
transcript_id: transcriptId,
},
},
body: {
long_summary: newSummary,
},
});
if (props.onUpdate) {
props.onUpdate(newSummary);
@@ -60,7 +60,7 @@ export default function FinalSummary(props: FinalSummaryProps) {
console.log("Updated long summary:", updatedTranscript);
} catch (err) {
console.error("Failed to update long summary:", err);
setError(err, "Failed to update long summary.");
setError(err as Error, "Failed to update long summary.");
}
};
@@ -114,7 +114,12 @@ export default function FinalSummary(props: FinalSummaryProps) {
<Button onClick={onDiscardClick} variant="ghost">
Cancel
</Button>
<Button onClick={onSaveClick}>Save</Button>
<Button
onClick={onSaveClick}
disabled={updateTranscriptMutation.isPending}
>
Save
</Button>
</Flex>
)}
{!isEditMode && (

46
www/app/(app)/transcripts/[transcriptId]/page.tsx
View File

@@ -1,32 +1,38 @@
"use client";
import Modal from "../modal";
import useTranscript from "../useTranscript";
import useTopics from "../useTopics";
import useWaveform from "../useWaveform";
import useMp3 from "../useMp3";
import { TopicList } from "./_components/TopicList";
import { Topic } from "../webSocketTypes";
import React, { useEffect, useState } from "react";
import React, { useEffect, useState, use } from "react";
import FinalSummary from "./finalSummary";
import TranscriptTitle from "../transcriptTitle";
import Player from "../player";
import { useRouter } from "next/navigation";
import { Box, Flex, Grid, GridItem, Skeleton, Text } from "@chakra-ui/react";
import { useTranscriptGet } from "../../../lib/apiHooks";
import { TranscriptStatus } from "../../../lib/transcript";
type TranscriptDetails = {
params: {
params: Promise<{
transcriptId: string;
};
}>;
};
export default function TranscriptDetails(details: TranscriptDetails) {
const transcriptId = details.params.transcriptId;
const params = use(details.params);
const transcriptId = params.transcriptId;
const router = useRouter();
const statusToRedirect = ["idle", "recording", "processing"];
const statusToRedirect = [
"idle",
"recording",
"processing",
] satisfies TranscriptStatus[] as TranscriptStatus[];
const transcript = useTranscript(transcriptId);
const transcriptStatus = transcript.response?.status;
const waiting = statusToRedirect.includes(transcriptStatus || "");
const transcript = useTranscriptGet(transcriptId);
const waiting =
transcript.data && statusToRedirect.includes(transcript.data.status);
const mp3 = useMp3(transcriptId, waiting);
const topics = useTopics(transcriptId);
@@ -38,7 +44,7 @@ export default function TranscriptDetails(details: TranscriptDetails) {
useEffect(() => {
if (waiting) {
const newUrl = "/transcripts/" + details.params.transcriptId + "/record";
const newUrl = "/transcripts/" + params.transcriptId + "/record";
// Shallow redirection does not work on NextJS 13
// https://github.com/vercel/next.js/discussions/48110
// https://github.com/vercel/next.js/discussions/49540
@@ -56,7 +62,7 @@ export default function TranscriptDetails(details: TranscriptDetails) {
);
}
if (transcript?.loading || topics?.loading) {
if (transcript?.isLoading || topics?.loading) {
return <Modal title="Loading" text={"Loading transcript..."} />;
}
@@ -86,7 +92,7 @@ export default function TranscriptDetails(details: TranscriptDetails) {
useActiveTopic={useActiveTopic}
waveform={waveform.waveform}
media={mp3.media}
mediaDuration={transcript.response.duration}
mediaDuration={transcript.data?.duration || null}
/>
) : !mp3.loading && (waveform.error || mp3.error) ? (
<Box p={4} bg="red.100" borderRadius="md">
@@ -116,10 +122,10 @@ export default function TranscriptDetails(details: TranscriptDetails) {
<Flex direction="column" gap={0}>
<Flex alignItems="center" gap={2}>
<TranscriptTitle
title={transcript.response.title || "Unnamed Transcript"}
title={transcript.data?.title || "Unnamed Transcript"}
transcriptId={transcriptId}
onUpdate={(newTitle) => {
transcript.reload();
transcript.refetch().then(() => {});
}}
/>
</Flex>
@@ -136,23 +142,23 @@ export default function TranscriptDetails(details: TranscriptDetails) {
useActiveTopic={useActiveTopic}
autoscroll={false}
transcriptId={transcriptId}
status={transcript.response?.status}
status={transcript.data?.status || null}
currentTranscriptText=""
/>
{transcript.response && topics.topics ? (
{transcript.data && topics.topics ? (
<>
<FinalSummary
transcriptResponse={transcript.response}
transcriptResponse={transcript.data}
topicsResponse={topics.topics}
onUpdate={(newSummary) => {
transcript.reload();
onUpdate={() => {
transcript.refetch();
}}
/>
</>
) : (
<Flex justify={"center"} alignItems={"center"} h={"100%"}>
<div className="flex flex-col h-full justify-center content-center">
{transcript.response.status == "processing" ? (
{transcript?.data?.status == "processing" ? (
<Text>Loading Transcript</Text>
) : (
<Text>

30
www/app/(app)/transcripts/[transcriptId]/record/page.tsx
View File

@@ -1,8 +1,7 @@
"use client";
import { useEffect, useState } from "react";
import { useEffect, useState, use } from "react";
import Recorder from "../../recorder";
import { TopicList } from "../_components/TopicList";
import useTranscript from "../../useTranscript";
import { useWebSockets } from "../../useWebSockets";
import { Topic } from "../../webSocketTypes";
import { lockWakeState, releaseWakeState } from "../../../../lib/wakeLock";
@@ -11,26 +10,29 @@ import useMp3 from "../../useMp3";
import WaveformLoading from "../../waveformLoading";
import { Box, Text, Grid, Heading, VStack, Flex } from "@chakra-ui/react";
import LiveTrancription from "../../liveTranscription";
import { useTranscriptGet } from "../../../../lib/apiHooks";
import { TranscriptStatus } from "../../../../lib/transcript";
type TranscriptDetails = {
params: {
params: Promise<{
transcriptId: string;
};
}>;
};
const TranscriptRecord = (details: TranscriptDetails) => {
const transcript = useTranscript(details.params.transcriptId);
const params = use(details.params);
const transcript = useTranscriptGet(params.transcriptId);
const [transcriptStarted, setTranscriptStarted] = useState(false);
const useActiveTopic = useState<Topic | null>(null);
const webSockets = useWebSockets(details.params.transcriptId);
const webSockets = useWebSockets(params.transcriptId);
const mp3 = useMp3(details.params.transcriptId, true);
const mp3 = useMp3(params.transcriptId, true);
const router = useRouter();
const [status, setStatus] = useState(
webSockets.status.value || transcript.response?.status || "idle",
const [status, setStatus] = useState<TranscriptStatus>(
webSockets.status?.value || transcript.data?.status || "idle",
);
useEffect(() => {
@@ -41,15 +43,15 @@ const TranscriptRecord = (details: TranscriptDetails) => {
useEffect(() => {
//TODO HANDLE ERROR STATUS BETTER
const newStatus =
webSockets.status.value || transcript.response?.status || "idle";
webSockets.status?.value || transcript.data?.status || "idle";
setStatus(newStatus);
if (newStatus && (newStatus == "ended" || newStatus == "error")) {
console.log(newStatus, "redirecting");
const newUrl = "/transcripts/" + details.params.transcriptId;
const newUrl = "/transcripts/" + params.transcriptId;
router.replace(newUrl);
}
}, [webSockets.status.value, transcript.response?.status]);
}, [webSockets.status?.value, transcript.data?.status]);
useEffect(() => {
if (webSockets.waveform && webSockets.waveform) mp3.getNow();
@@ -74,7 +76,7 @@ const TranscriptRecord = (details: TranscriptDetails) => {
<WaveformLoading />
) : (
// todo: only start recording animation when you get "recorded" status
<Recorder transcriptId={details.params.transcriptId} status={status} />
<Recorder transcriptId={params.transcriptId} status={status} />
)}
<VStack
align={"left"}
@@ -97,7 +99,7 @@ const TranscriptRecord = (details: TranscriptDetails) => {
topics={webSockets.topics}
useActiveTopic={useActiveTopic}
autoscroll={true}
transcriptId={details.params.transcriptId}
transcriptId={params.transcriptId}
status={status}
currentTranscriptText={webSockets.accumulatedText}
/>

36
www/app/(app)/transcripts/[transcriptId]/upload/page.tsx
View File

@@ -1,33 +1,40 @@
"use client";
import { useEffect, useState } from "react";
import useTranscript from "../../useTranscript";
import { useEffect, useState, use } from "react";
import { useWebSockets } from "../../useWebSockets";
import { lockWakeState, releaseWakeState } from "../../../../lib/wakeLock";
import { useRouter } from "next/navigation";
import useMp3 from "../../useMp3";
import { Center, VStack, Text, Heading, Button } from "@chakra-ui/react";
import FileUploadButton from "../../fileUploadButton";
import { useTranscriptGet } from "../../../../lib/apiHooks";
type TranscriptUpload = {
params: {
params: Promise<{
transcriptId: string;
};
}>;
};
const TranscriptUpload = (details: TranscriptUpload) => {
const transcript = useTranscript(details.params.transcriptId);
const params = use(details.params);
const transcript = useTranscriptGet(params.transcriptId);
const [transcriptStarted, setTranscriptStarted] = useState(false);
const webSockets = useWebSockets(details.params.transcriptId);
const webSockets = useWebSockets(params.transcriptId);
const mp3 = useMp3(details.params.transcriptId, true);
const mp3 = useMp3(params.transcriptId, true);
const router = useRouter();
const [status, setStatus] = useState(
webSockets.status.value || transcript.response?.status || "idle",
const [status_, setStatus] = useState(
webSockets.status?.value || transcript.data?.status || "idle",
);
// status is obviously done if we have transcript
const status =
!transcript.isLoading && transcript.data?.status === "ended"
? transcript.data?.status
: status_;
useEffect(() => {
if (!transcriptStarted && webSockets.transcriptTextLive.length !== 0)
setTranscriptStarted(true);
@@ -35,16 +42,19 @@ const TranscriptUpload = (details: TranscriptUpload) => {
useEffect(() => {
//TODO HANDLE ERROR STATUS BETTER
// TODO deprecate webSockets.status.value / depend on transcript.response?.status from query lib
const newStatus =
webSockets.status.value || transcript.response?.status || "idle";
transcript.data?.status === "ended"
? "ended"
: webSockets.status?.value || transcript.data?.status || "idle";
setStatus(newStatus);
if (newStatus && (newStatus == "ended" || newStatus == "error")) {
console.log(newStatus, "redirecting");
const newUrl = "/transcripts/" + details.params.transcriptId;
const newUrl = "/transcripts/" + params.transcriptId;
router.replace(newUrl);
}
}, [webSockets.status.value, transcript.response?.status]);
}, [webSockets.status?.value, transcript.data?.status]);
useEffect(() => {
if (webSockets.waveform && webSockets.waveform) mp3.getNow();
@@ -75,7 +85,7 @@ const TranscriptUpload = (details: TranscriptUpload) => {
Please select the file, supported formats: .mp3, m4a, .wav,
.mp4, .mov or .webm
</Text>
<FileUploadButton transcriptId={details.params.transcriptId} />
<FileUploadButton transcriptId={params.transcriptId} />
</>
)}
{status && status == "uploaded" && (

46
www/app/(app)/transcripts/createTranscript.ts
View File

@@ -1,45 +1,33 @@
import { useEffect, useState } from "react";
import type { components } from "../../reflector-api";
import { useTranscriptCreate } from "../../lib/apiHooks";
import { useError } from "../../(errors)/errorContext";
import { CreateTranscript, GetTranscript } from "../../api";
import useApi from "../../lib/useApi";
type CreateTranscript = components["schemas"]["CreateTranscript"];
type GetTranscript = components["schemas"]["GetTranscript"];
type UseCreateTranscript = {
transcript: GetTranscript | null;
loading: boolean;
error: Error | null;
create: (transcriptCreationDetails: CreateTranscript) => void;
create: (transcriptCreationDetails: CreateTranscript) => Promise<void>;
};
const useCreateTranscript = (): UseCreateTranscript => {
const [transcript, setTranscript] = useState<GetTranscript | null>(null);
const [loading, setLoading] = useState<boolean>(false);
const [error, setErrorState] = useState<Error | null>(null);
const { setError } = useError();
const api = useApi();
const createMutation = useTranscriptCreate();
const create = (transcriptCreationDetails: CreateTranscript) => {
if (loading || !api) return;
const create = async (transcriptCreationDetails: CreateTranscript) => {
if (createMutation.isPending) return;
setLoading(true);
api
.v1TranscriptsCreate({ requestBody: transcriptCreationDetails })
.then((transcript) => {
setTranscript(transcript);
setLoading(false);
})
.catch((err) => {
setError(
err,
"There was an issue creating a transcript, please try again.",
);
setErrorState(err);
setLoading(false);
});
await createMutation.mutateAsync({
body: transcriptCreationDetails,
});
};
return { transcript, loading, error, create };
return {
transcript: createMutation.data || null,
loading: createMutation.isPending,
error: createMutation.error as Error | null,
create,
};
};
export default useCreateTranscript;

64
www/app/(app)/transcripts/fileUploadButton.tsx
View File

@@ -1,6 +1,7 @@
import React, { useState } from "react";
import useApi from "../../lib/useApi";
import { useTranscriptUploadAudio } from "../../lib/apiHooks";
import { Button, Spinner } from "@chakra-ui/react";
import { useError } from "../../(errors)/errorContext";
type FileUploadButton = {
transcriptId: string;
@@ -8,13 +9,16 @@ type FileUploadButton = {
export default function FileUploadButton(props: FileUploadButton) {
const fileInputRef = React.useRef<HTMLInputElement>(null);
const api = useApi();
const uploadMutation = useTranscriptUploadAudio();
const { setError } = useError();
const [progress, setProgress] = useState(0);
const triggerFileUpload = () => {
fileInputRef.current?.click();
};
const handleFileUpload = (event: React.ChangeEvent<HTMLInputElement>) => {
const handleFileUpload = async (
event: React.ChangeEvent<HTMLInputElement>,
) => {
const file = event.target.files?.[0];
if (file) {
@@ -24,37 +28,45 @@ export default function FileUploadButton(props: FileUploadButton) {
let start = 0;
let uploadedSize = 0;
api?.httpRequest.config.interceptors.request.use((request) => {
request.onUploadProgress = (progressEvent) => {
const currentProgress = Math.floor(
((uploadedSize + progressEvent.loaded) / file.size) * 100,
);
setProgress(currentProgress);
};
return request;
});
const uploadNextChunk = async () => {
if (chunkNumber == totalChunks) return;
if (chunkNumber == totalChunks) {
setProgress(0);
return;
}
const chunkSize = Math.min(maxChunkSize, file.size - start);
const end = start + chunkSize;
const chunk = file.slice(start, end);
await api?.v1TranscriptRecordUpload({
transcriptId: props.transcriptId,
formData: {
chunk,
},
chunkNumber,
totalChunks,
});
try {
const formData = new FormData();
formData.append("chunk", chunk);
uploadedSize += chunkSize;
chunkNumber++;
start = end;
await uploadMutation.mutateAsync({
params: {
path: {
transcript_id: props.transcriptId,
},
query: {
chunk_number: chunkNumber,
total_chunks: totalChunks,
},
},
body: formData as any,
});
uploadNextChunk();
uploadedSize += chunkSize;
const currentProgress = Math.floor((uploadedSize / file.size) * 100);
setProgress(currentProgress);
chunkNumber++;
start = end;
await uploadNextChunk();
} catch (error) {
setError(error as Error, "Failed to upload file");
setProgress(0);
}
};
uploadNextChunk();

46
www/app/(app)/transcripts/new/page.tsx
View File

@@ -9,33 +9,25 @@ import { useRouter } from "next/navigation";
import useCreateTranscript from "../createTranscript";
import SelectSearch from "react-select-search";
import { supportedLanguages } from "../../../supportedLanguages";
import useSessionStatus from "../../../lib/useSessionStatus";
import { featureEnabled } from "../../../domainContext";
import { signIn } from "next-auth/react";
import {
Flex,
Box,
Spinner,
Heading,
Button,
Card,
Center,
Link,
CardBody,
Stack,
Text,
Icon,
Grid,
IconButton,
Spacer,
Menu,
Tooltip,
Input,
} from "@chakra-ui/react";
import { useAuth } from "../../../lib/AuthProvider";
import { featureEnabled } from "../../../lib/features";
const TranscriptCreate = () => {
const isClient = typeof window !== "undefined";
const router = useRouter();
const { isLoading, isAuthenticated } = useSessionStatus();
const auth = useAuth();
const isAuthenticated = auth.status === "authenticated";
const isAuthRefreshing = auth.status === "refreshing";
const isLoading = auth.status === "loading";
const requireLogin = featureEnabled("requireLogin");
const [name, setName] = useState<string>("");
@@ -54,20 +46,32 @@ const TranscriptCreate = () => {
const [loadingUpload, setLoadingUpload] = useState(false);
const getTargetLanguage = () => {
if (targetLanguage === "NOTRANSLATION") return;
if (targetLanguage === "NOTRANSLATION") return undefined;
return targetLanguage;
};
const send = () => {
if (loadingRecord || createTranscript.loading || permissionDenied) return;
setLoadingRecord(true);
createTranscript.create({ name, target_language: getTargetLanguage() });
const targetLang = getTargetLanguage();
createTranscript.create({
name,
source_language: "en",
target_language: targetLang || "en",
source_kind: "live",
});
};
const uploadFile = () => {
if (loadingUpload || createTranscript.loading || permissionDenied) return;
setLoadingUpload(true);
createTranscript.create({ name, target_language: getTargetLanguage() });
const targetLang = getTargetLanguage();
createTranscript.create({
name,
source_language: "en",
target_language: targetLang || "en",
source_kind: "file",
});
};
useEffect(() => {
@@ -132,8 +136,8 @@ const TranscriptCreate = () => {
<Center>
{isLoading ? (
<Spinner />
) : requireLogin && !isAuthenticated ? (
<Button onClick={() => signIn("authentik")}>Log in</Button>
) : requireLogin && !isAuthenticated && !isAuthRefreshing ? (
<Button onClick={() => auth.signIn("authentik")}>Log in</Button>
) : (
<Flex
rounded="xl"
@@ -170,7 +174,7 @@ const TranscriptCreate = () => {
placeholder="Choose your language"
/>
</Box>
{isClient && !loading ? (
{!loading ? (
permissionOk ? (
<Spacer />
) : permissionDenied ? (

10
www/app/(app)/transcripts/player.tsx
View File

@@ -5,7 +5,9 @@ import RegionsPlugin from "wavesurfer.js/dist/plugins/regions.esm.js";
import { formatTime, formatTimeMs } from "../../lib/time";
import { Topic } from "./webSocketTypes";
import { AudioWaveform } from "../../api";
import type { components } from "../../reflector-api";
type AudioWaveform = components["schemas"]["AudioWaveform"];
import { waveSurferStyles } from "../../styles/recorder";
import { Box, Flex, IconButton } from "@chakra-ui/react";
import { LuPause, LuPlay } from "react-icons/lu";
@@ -18,7 +20,7 @@ type PlayerProps = {
];
waveform: AudioWaveform;
media: HTMLMediaElement;
mediaDuration: number;
mediaDuration: number | null;
};
export default function Player(props: PlayerProps) {
@@ -50,7 +52,9 @@ export default function Player(props: PlayerProps) {
container: waveformRef.current,
peaks: [props.waveform.data],
height: "auto",
duration: Math.floor(props.mediaDuration / 1000),
duration: props.mediaDuration
? Math.floor(props.mediaDuration / 1000)
: undefined,
media: props.media,
...waveSurferStyles.playerSettings,

4
www/app/(app)/transcripts/recorder.tsx
View File

@@ -6,16 +6,16 @@ import RecordPlugin from "../../lib/custom-plugins/record";
import { formatTime, formatTimeMs } from "../../lib/time";
import { waveSurferStyles } from "../../styles/recorder";
import { useError } from "../../(errors)/errorContext";
import FileUploadButton from "./fileUploadButton";
import useWebRTC from "./useWebRTC";
import useAudioDevice from "./useAudioDevice";
import { Box, Flex, IconButton, Menu, RadioGroup } from "@chakra-ui/react";
import { LuScreenShare, LuMic, LuPlay, LuCircleStop } from "react-icons/lu";
import { RECORD_A_MEETING_URL } from "../../api/urls";
import { TranscriptStatus } from "../../lib/transcript";
type RecorderProps = {
transcriptId: string;
status: string;
status: TranscriptStatus;
};
export default function Recorder(props: RecorderProps) {

50
www/app/(app)/transcripts/shareAndPrivacy.tsx
View File

@@ -1,8 +1,10 @@
import { useEffect, useState } from "react";
import { featureEnabled } from "../../domainContext";
import { ShareMode, toShareMode } from "../../lib/shareMode";
import { GetTranscript, GetTranscriptTopic, UpdateTranscript } from "../../api";
import type { components } from "../../reflector-api";
type GetTranscript = components["schemas"]["GetTranscript"];
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
type UpdateTranscript = components["schemas"]["UpdateTranscript"];
import {
Box,
Flex,
@@ -15,12 +17,13 @@ import {
createListCollection,
} from "@chakra-ui/react";
import { LuShare2 } from "react-icons/lu";
import useApi from "../../lib/useApi";
import useSessionUser from "../../lib/useSessionUser";
import { CustomSession } from "../../lib/types";
import { useTranscriptUpdate } from "../../lib/apiHooks";
import ShareLink from "./shareLink";
import ShareCopy from "./shareCopy";
import ShareZulip from "./shareZulip";
import { useAuth } from "../../lib/AuthProvider";
import { featureEnabled } from "../../lib/features";
type ShareAndPrivacyProps = {
finalSummaryRef: any;
@@ -50,12 +53,9 @@ export default function ShareAndPrivacy(props: ShareAndPrivacyProps) {
);
const [shareLoading, setShareLoading] = useState(false);
const requireLogin = featureEnabled("requireLogin");
const api = useApi();
const updateTranscriptMutation = useTranscriptUpdate();
const updateShareMode = async (selectedValue: string) => {
if (!api)
throw new Error("ShareLink's API should always be ready at this point");
const selectedOption = shareOptionsData.find(
(option) => option.value === selectedValue,
);
@@ -67,19 +67,27 @@ export default function ShareAndPrivacy(props: ShareAndPrivacyProps) {
share_mode: selectedValue as "public" | "semi-private" | "private",
};
const updatedTranscript = await api.v1TranscriptUpdate({
transcriptId: props.transcriptResponse.id,
requestBody,
});
setShareMode(
shareOptionsData.find(
(option) => option.value === updatedTranscript.share_mode,
) || shareOptionsData[0],
);
setShareLoading(false);
try {
const updatedTranscript = await updateTranscriptMutation.mutateAsync({
params: {
path: { transcript_id: props.transcriptResponse.id },
},
body: requestBody,
});
setShareMode(
shareOptionsData.find(
(option) => option.value === updatedTranscript.share_mode,
) || shareOptionsData[0],
);
} catch (err) {
console.error("Failed to update share mode:", err);
} finally {
setShareLoading(false);
}
};
const userId = useSessionUser().id;
const auth = useAuth();
const userId = auth.status === "authenticated" ? auth.user?.id : null;
useEffect(() => {
setIsOwner(!!(requireLogin && userId === props.transcriptResponse.user_id));
@@ -124,7 +132,7 @@ export default function ShareAndPrivacy(props: ShareAndPrivacyProps) {
"This transcript is public. Everyone can access it."}
</Text>
{isOwner && api && (
{isOwner && (
<Select.Root
key={shareMode.value}
value={[shareMode.value || ""]}

4
www/app/(app)/transcripts/shareCopy.tsx
View File

@@ -1,5 +1,7 @@
import { useState } from "react";
import { GetTranscript, GetTranscriptTopic } from "../../api";
import type { components } from "../../reflector-api";
type GetTranscript = components["schemas"]["GetTranscript"];
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
import { Button, BoxProps, Box } from "@chakra-ui/react";
type ShareCopyProps = {

3
www/app/(app)/transcripts/shareLink.tsx
View File

@@ -1,8 +1,9 @@
import React, { useState, useRef, useEffect, use } from "react";
import { featureEnabled } from "../../domainContext";
import { Button, Flex, Input, Text } from "@chakra-ui/react";
import QRCode from "react-qr-code";
import { featureEnabled } from "../../lib/features";
type ShareLinkProps = {
transcriptId: string;
};

169
www/app/(app)/transcripts/shareZulip.tsx
View File

@@ -1,6 +1,8 @@
import { useState, useEffect, useMemo } from "react";
import { featureEnabled } from "../../domainContext";
import { GetTranscript, GetTranscriptTopic } from "../../api";
import type { components } from "../../reflector-api";
type GetTranscript = components["schemas"]["GetTranscript"];
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
import {
BoxProps,
Button,
@@ -12,12 +14,16 @@ import {
Checkbox,
Combobox,
Spinner,
Portal,
useFilter,
useListCollection,
createListCollection,
} from "@chakra-ui/react";
import { TbBrandZulip } from "react-icons/tb";
import useApi from "../../lib/useApi";
import {
useZulipStreams,
useZulipTopics,
useTranscriptPostToZulip,
} from "../../lib/apiHooks";
import { featureEnabled } from "../../lib/features";
type ShareZulipProps = {
transcriptResponse: GetTranscript;
@@ -30,104 +36,77 @@ interface Stream {
name: string;
}
interface Topic {
name: string;
}
export default function ShareZulip(props: ShareZulipProps & BoxProps) {
const [showModal, setShowModal] = useState(false);
const [stream, setStream] = useState<string | undefined>(undefined);
const [selectedStreamId, setSelectedStreamId] = useState<number | null>(null);
const [topic, setTopic] = useState<string | undefined>(undefined);
const [includeTopics, setIncludeTopics] = useState(false);
const [isLoading, setIsLoading] = useState(true);
const [streams, setStreams] = useState<Stream[]>([]);
const [topics, setTopics] = useState<Topic[]>([]);
const api = useApi();
const { contains } = useFilter({ sensitivity: "base" });
const {
collection: streamItemsCollection,
filter: streamItemsFilter,
set: streamItemsSet,
} = useListCollection({
initialItems: [] as { label: string; value: string }[],
filter: contains,
});
const { data: streams = [], isLoading: isLoadingStreams } = useZulipStreams();
const { data: topics = [] } = useZulipTopics(selectedStreamId);
const postToZulipMutation = useTranscriptPostToZulip();
const {
collection: topicItemsCollection,
filter: topicItemsFilter,
set: topicItemsSet,
} = useListCollection({
initialItems: [] as { label: string; value: string }[],
filter: contains,
});
const streamItems = useMemo(() => {
return streams.map((stream: Stream) => ({
label: stream.name,
value: stream.name,
}));
}, [streams]);
const topicItems = useMemo(() => {
return topics.map(({ name }) => ({
label: name,
value: name,
}));
}, [topics]);
const streamCollection = useMemo(
() =>
createListCollection({
items: streamItems,
}),
[streamItems],
);
const topicCollection = useMemo(
() =>
createListCollection({
items: topicItems,
}),
[topicItems],
);
// Update selected stream ID when stream changes
useEffect(() => {
const fetchZulipStreams = async () => {
if (!api) return;
try {
const response = await api.v1ZulipGetStreams();
setStreams(response);
streamItemsSet(
response.map((stream) => ({
label: stream.name,
value: stream.name,
})),
);
setIsLoading(false);
} catch (error) {
console.error("Error fetching Zulip streams:", error);
}
};
fetchZulipStreams();
}, [!api]);
useEffect(() => {
const fetchZulipTopics = async () => {
if (!api || !stream) return;
try {
const selectedStream = streams.find((s) => s.name === stream);
if (selectedStream) {
const response = await api.v1ZulipGetTopics({
streamId: selectedStream.stream_id,
});
setTopics(response);
topicItemsSet(
response.map((topic) => ({
label: topic.name,
value: topic.name,
})),
);
} else {
topicItemsSet([]);
}
} catch (error) {
console.error("Error fetching Zulip topics:", error);
}
};
fetchZulipTopics();
}, [stream, streams, api]);
if (stream && streams) {
const selectedStream = streams.find((s: Stream) => s.name === stream);
setSelectedStreamId(selectedStream ? selectedStream.stream_id : null);
} else {
setSelectedStreamId(null);
}
}, [stream, streams]);
const handleSendToZulip = async () => {
if (!api || !props.transcriptResponse) return;
if (!props.transcriptResponse) return;
if (stream && topic) {
try {
await api.v1TranscriptPostToZulip({
transcriptId: props.transcriptResponse.id,
stream,
topic,
includeTopics,
await postToZulipMutation.mutateAsync({
params: {
path: {
transcript_id: props.transcriptResponse.id,
},
query: {
stream,
topic,
include_topics: includeTopics,
},
},
});
setShowModal(false);
} catch (error) {
console.log(error);
console.error("Error posting to Zulip:", error);
}
}
};
@@ -155,7 +134,7 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
</Dialog.CloseTrigger>
</Dialog.Header>
<Dialog.Body>
{isLoading ? (
{isLoadingStreams ? (
<Flex justify="center" py={8}>
<Spinner />
</Flex>
@@ -178,15 +157,12 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
<Flex align="center" gap={2}>
<Text>#</Text>
<Combobox.Root
collection={streamItemsCollection}
collection={streamCollection}
value={stream ? [stream] : []}
onValueChange={(e) => {
setTopic(undefined);
setStream(e.value[0]);
}}
onInputValueChange={(e) =>
streamItemsFilter(e.inputValue)
}
openOnClick={true}
positioning={{
strategy: "fixed",
@@ -203,7 +179,7 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
<Combobox.Positioner>
<Combobox.Content>
<Combobox.Empty>No streams found</Combobox.Empty>
{streamItemsCollection.items.map((item) => (
{streamItems.map((item) => (
<Combobox.Item key={item.value} item={item}>
{item.label}
</Combobox.Item>
@@ -219,12 +195,9 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
<Flex align="center" gap={2}>
<Text visibility="hidden">#</Text>
<Combobox.Root
collection={topicItemsCollection}
collection={topicCollection}
value={topic ? [topic] : []}
onValueChange={(e) => setTopic(e.value[0])}
onInputValueChange={(e) =>
topicItemsFilter(e.inputValue)
}
openOnClick
selectionBehavior="replace"
skipAnimationOnMount={true}
@@ -244,7 +217,7 @@ export default function ShareZulip(props: ShareZulipProps & BoxProps) {
<Combobox.Positioner>
<Combobox.Content>
<Combobox.Empty>No topics found</Combobox.Empty>
{topicItemsCollection.items.map((item) => (
{topicItems.map((item) => (
<Combobox.Item key={item.value} item={item}>
{item.label}
<Combobox.ItemIndicator />

21
www/app/(app)/transcripts/transcriptTitle.tsx
View File

@@ -1,6 +1,8 @@
import { useState } from "react";
import { UpdateTranscript } from "../../api";
import useApi from "../../lib/useApi";
import type { components } from "../../reflector-api";
type UpdateTranscript = components["schemas"]["UpdateTranscript"];
import { useTranscriptUpdate } from "../../lib/apiHooks";
import { Heading, IconButton, Input, Flex, Spacer } from "@chakra-ui/react";
import { LuPen } from "react-icons/lu";
@@ -14,24 +16,27 @@ const TranscriptTitle = (props: TranscriptTitle) => {
const [displayedTitle, setDisplayedTitle] = useState(props.title);
const [preEditTitle, setPreEditTitle] = useState(props.title);
const [isEditing, setIsEditing] = useState(false);
const api = useApi();
const updateTranscriptMutation = useTranscriptUpdate();
const updateTitle = async (newTitle: string, transcriptId: string) => {
if (!api) return;
try {
const requestBody: UpdateTranscript = {
title: newTitle,
};
const updatedTranscript = await api?.v1TranscriptUpdate({
transcriptId,
requestBody,
await updateTranscriptMutation.mutateAsync({
params: {
path: { transcript_id: transcriptId },
},
body: requestBody,
});
if (props.onUpdate) {
props.onUpdate(newTitle);
}
console.log("Updated transcript:", updatedTranscript);
console.log("Updated transcript title:", newTitle);
} catch (err) {
console.error("Failed to update transcript:", err);
// Revert title on error
setDisplayedTitle(preEditTitle);
}
};

113
www/app/(app)/transcripts/useMp3.ts
View File

@@ -1,6 +1,7 @@
import { useContext, useEffect, useState } from "react";
import { DomainContext } from "../../domainContext";
import getApi from "../../lib/useApi";
import { useEffect, useState } from "react";
import { useTranscriptGet } from "../../lib/apiHooks";
import { useAuth } from "../../lib/AuthProvider";
import { API_URL } from "../../lib/apiClient";
export type Mp3Response = {
media: HTMLMediaElement | null;
@@ -17,14 +18,16 @@ const useMp3 = (transcriptId: string, waiting?: boolean): Mp3Response => {
const [audioLoadingError, setAudioLoadingError] = useState<null | string>(
null,
);
const [transcriptMetadataLoading, setTranscriptMetadataLoading] =
useState<boolean>(true);
const [transcriptMetadataLoadingError, setTranscriptMetadataLoadingError] =
useState<string | null>(null);
const [audioDeleted, setAudioDeleted] = useState<boolean | null>(null);
const api = getApi();
const { api_url } = useContext(DomainContext);
const accessTokenInfo = api?.httpRequest?.config?.TOKEN;
const auth = useAuth();
const accessTokenInfo =
auth.status === "authenticated" ? auth.accessToken : null;
const {
data: transcript,
isLoading: transcriptMetadataLoading,
error: transcriptError,
} = useTranscriptGet(later ? null : transcriptId);
const [serviceWorker, setServiceWorker] =
useState<ServiceWorkerRegistration | null>(null);
@@ -52,72 +55,50 @@ const useMp3 = (transcriptId: string, waiting?: boolean): Mp3Response => {
}, [navigator.serviceWorker, !serviceWorker, accessTokenInfo]);
useEffect(() => {
if (!transcriptId || !api || later) return;
if (!transcriptId || later || !transcript) return;
let stopped = false;
let audioElement: HTMLAudioElement | null = null;
let handleCanPlay: (() => void) | null = null;
let handleError: (() => void) | null = null;
setTranscriptMetadataLoading(true);
setAudioLoading(true);
// First fetch transcript info to check if audio is deleted
api
.v1TranscriptGet({ transcriptId })
.then((transcript) => {
if (stopped) {
return;
}
const deleted = transcript.audio_deleted || false;
setAudioDeleted(deleted);
const deleted = transcript.audio_deleted || false;
setAudioDeleted(deleted);
setTranscriptMetadataLoadingError(null);
if (deleted) {
// Audio is deleted, don't attempt to load it
setMedia(null);
setAudioLoadingError(null);
setAudioLoading(false);
return;
}
if (deleted) {
// Audio is deleted, don't attempt to load it
setMedia(null);
setAudioLoadingError(null);
setAudioLoading(false);
return;
}
// Audio is not deleted, proceed to load it
audioElement = document.createElement("audio");
audioElement.src = `${API_URL}/v1/transcripts/${transcriptId}/audio/mp3`;
audioElement.crossOrigin = "anonymous";
audioElement.preload = "auto";
// Audio is not deleted, proceed to load it
audioElement = document.createElement("audio");
audioElement.src = `${api_url}/v1/transcripts/${transcriptId}/audio/mp3`;
audioElement.crossOrigin = "anonymous";
audioElement.preload = "auto";
handleCanPlay = () => {
if (stopped) return;
setAudioLoading(false);
setAudioLoadingError(null);
};
handleCanPlay = () => {
if (stopped) return;
setAudioLoading(false);
setAudioLoadingError(null);
};
handleError = () => {
if (stopped) return;
setAudioLoading(false);
setAudioLoadingError("Failed to load audio");
};
handleError = () => {
if (stopped) return;
setAudioLoading(false);
setAudioLoadingError("Failed to load audio");
};
audioElement.addEventListener("canplay", handleCanPlay);
audioElement.addEventListener("error", handleError);
audioElement.addEventListener("canplay", handleCanPlay);
audioElement.addEventListener("error", handleError);
if (!stopped) {
setMedia(audioElement);
}
})
.catch((error) => {
if (stopped) return;
console.error("Failed to fetch transcript:", error);
setAudioDeleted(null);
setTranscriptMetadataLoadingError(error.message);
setAudioLoading(false);
})
.finally(() => {
if (stopped) return;
setTranscriptMetadataLoading(false);
});
if (!stopped) {
setMedia(audioElement);
}
return () => {
stopped = true;
@@ -128,14 +109,18 @@ const useMp3 = (transcriptId: string, waiting?: boolean): Mp3Response => {
if (handleError) audioElement.removeEventListener("error", handleError);
}
};
}, [transcriptId, api, later, api_url]);
}, [transcriptId, transcript, later]);
const getNow = () => {
setLater(false);
};
const loading = audioLoading || transcriptMetadataLoading;
const error = audioLoadingError || transcriptMetadataLoadingError;
const error =
audioLoadingError ||
(transcriptError
? (transcriptError as any).message || String(transcriptError)
: null);
return { media, loading, error, getNow, audioDeleted };
};

74
www/app/(app)/transcripts/useParticipants.ts
View File

@@ -1,8 +1,6 @@
import { useEffect, useState } from "react";
import { Participant } from "../../api";
import { useError } from "../../(errors)/errorContext";
import useApi from "../../lib/useApi";
import { shouldShowError } from "../../lib/errorUtils";
import type { components } from "../../reflector-api";
type Participant = components["schemas"]["Participant"];
import { useTranscriptParticipants } from "../../lib/apiHooks";
type ErrorParticipants = {
error: Error;
@@ -29,46 +27,38 @@ export type UseParticipants = (
) & { refetch: () => void };
const useParticipants = (transcriptId: string): UseParticipants => {
const [response, setResponse] = useState<Participant[] | null>(null);
const [loading, setLoading] = useState<boolean>(true);
const [error, setErrorState] = useState<Error | null>(null);
const { setError } = useError();
const api = useApi();
const [count, setCount] = useState(0);
const {
data: response,
isLoading: loading,
error,
refetch,
} = useTranscriptParticipants(transcriptId || null);
const refetch = () => {
if (!loading) {
setCount(count + 1);
setLoading(true);
setErrorState(null);
}
};
// Type-safe return based on state
if (error) {
return {
error: error as Error,
loading: false,
response: null,
refetch,
} satisfies ErrorParticipants & { refetch: () => void };
}
useEffect(() => {
if (!transcriptId || !api) return;
if (loading || !response) {
return {
response: response || null,
loading: true,
error: null,
refetch,
} satisfies LoadingParticipants & { refetch: () => void };
}
setLoading(true);
api
.v1TranscriptGetParticipants({ transcriptId })
.then((result) => {
setResponse(result);
setLoading(false);
console.debug("Participants Loaded:", result);
})
.catch((error) => {
const shouldShowHuman = shouldShowError(error);
if (shouldShowHuman) {
setError(error, "There was an error loading the participants");
} else {
setError(error);
}
setErrorState(error);
setResponse(null);
setLoading(false);
});
}, [transcriptId, !api, count]);
return { response, loading, error, refetch } as UseParticipants;
return {
response,
loading: false,
error: null,
refetch,
} satisfies SuccessParticipants & { refetch: () => void };
};
export default useParticipants;

123
www/app/(app)/transcripts/useSearchTranscripts.ts
View File

@@ -1,123 +0,0 @@
// this hook is not great, we want to substitute it with a proper state management solution that is also not re-invention
import { useEffect, useRef, useState } from "react";
import { SearchResult, SourceKind } from "../../api";
import useApi from "../../lib/useApi";
import {
PaginationPage,
paginationPageTo0Based,
} from "../browse/_components/Pagination";
interface SearchFilters {
roomIds: readonly string[] | null;
sourceKind: SourceKind | null;
}
const EMPTY_SEARCH_FILTERS: SearchFilters = {
roomIds: null,
sourceKind: null,
};
type UseSearchTranscriptsOptions = {
pageSize: number;
page: PaginationPage;
};
interface UseSearchTranscriptsReturn {
results: SearchResult[];
totalCount: number;
isLoading: boolean;
error: unknown;
reload: () => void;
}
function hashEffectFilters(filters: SearchFilters): string {
return JSON.stringify(filters);
}
export function useSearchTranscripts(
query: string = "",
filters: SearchFilters = EMPTY_SEARCH_FILTERS,
options: UseSearchTranscriptsOptions = {
pageSize: 20,
page: PaginationPage(1),
},
): UseSearchTranscriptsReturn {
const { pageSize, page } = options;
const [reloadCount, setReloadCount] = useState(0);
const api = useApi();
const abortControllerRef = useRef<AbortController>();
const [data, setData] = useState<{ results: SearchResult[]; total: number }>({
results: [],
total: 0,
});
const [error, setError] = useState<any>();
const [isLoading, setIsLoading] = useState(false);
const filterHash = hashEffectFilters(filters);
useEffect(() => {
if (!api) {
setData({ results: [], total: 0 });
setError(undefined);
setIsLoading(false);
return;
}
if (abortControllerRef.current) {
abortControllerRef.current.abort();
}
const abortController = new AbortController();
abortControllerRef.current = abortController;
const performSearch = async () => {
setIsLoading(true);
try {
const response = await api.v1TranscriptsSearch({
q: query || "",
limit: pageSize,
offset: paginationPageTo0Based(page) * pageSize,
roomId: filters.roomIds?.[0],
sourceKind: filters.sourceKind || undefined,
});
if (abortController.signal.aborted) return;
setData(response);
setError(undefined);
} catch (err: unknown) {
if ((err as Error).name === "AbortError") {
return;
}
if (abortController.signal.aborted) {
console.error("Aborted search but error", err);
return;
}
setError(err);
} finally {
if (!abortController.signal.aborted) {
setIsLoading(false);
}
}
};
performSearch().then(() => {});
return () => {
abortController.abort();
};
}, [api, query, page, filterHash, pageSize, reloadCount]);
return {
results: data.results,
totalCount: data.total,
isLoading,
error,
reload: () => setReloadCount(reloadCount + 1),
};
}

78
www/app/(app)/transcripts/useTopicWithWords.ts
View File

@@ -1,9 +1,8 @@
import { useEffect, useState } from "react";
import type { components } from "../../reflector-api";
import { useTranscriptTopicsWithWordsPerSpeaker } from "../../lib/apiHooks";
import { GetTranscriptTopicWithWordsPerSpeaker } from "../../api";
import { useError } from "../../(errors)/errorContext";
import useApi from "../../lib/useApi";
import { shouldShowError } from "../../lib/errorUtils";
type GetTranscriptTopicWithWordsPerSpeaker =
components["schemas"]["GetTranscriptTopicWithWordsPerSpeaker"];
type ErrorTopicWithWords = {
error: Error;
@@ -33,47 +32,40 @@ const useTopicWithWords = (
topicId: string | undefined,
transcriptId: string,
): UseTopicWithWords => {
const [response, setResponse] =
useState<GetTranscriptTopicWithWordsPerSpeaker | null>(null);
const [loading, setLoading] = useState<boolean>(false);
const [error, setErrorState] = useState<Error | null>(null);
const { setError } = useError();
const api = useApi();
const {
data: response,
isLoading: loading,
error,
refetch,
} = useTranscriptTopicsWithWordsPerSpeaker(
transcriptId || null,
topicId || null,
);
const [count, setCount] = useState(0);
if (error) {
return {
error: error as Error,
loading: false,
response: null,
refetch,
} satisfies ErrorTopicWithWords & { refetch: () => void };
}
const refetch = () => {
if (!loading) {
setCount(count + 1);
setLoading(true);
setErrorState(null);
}
};
if (loading || !response) {
return {
response: response || null,
loading: true,
error: false,
refetch,
} satisfies LoadingTopicWithWords & { refetch: () => void };
}
useEffect(() => {
if (!transcriptId || !topicId || !api) return;
setLoading(true);
api
.v1TranscriptGetTopicsWithWordsPerSpeaker({ transcriptId, topicId })
.then((result) => {
setResponse(result);
setLoading(false);
console.debug("Topics with words Loaded:", result);
})
.catch((error) => {
const shouldShowHuman = shouldShowError(error);
if (shouldShowHuman) {
setError(error, "There was an error loading the topics with words");
} else {
setError(error);
}
setErrorState(error);
});
}, [transcriptId, !api, topicId, count]);
return { response, loading, error, refetch } as UseTopicWithWords;
return {
response,
loading: false,
error: null,
refetch,
} satisfies SuccessTopicWithWords & { refetch: () => void };
};
export default useTopicWithWords;

42
www/app/(app)/transcripts/useTopics.ts
View File

@@ -1,10 +1,7 @@
import { useEffect, useState } from "react";
import { useTranscriptTopics } from "../../lib/apiHooks";
import type { components } from "../../reflector-api";
import { useError } from "../../(errors)/errorContext";
import { Topic } from "./webSocketTypes";
import useApi from "../../lib/useApi";
import { shouldShowError } from "../../lib/errorUtils";
import { GetTranscriptTopic } from "../../api";
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
type TranscriptTopics = {
topics: GetTranscriptTopic[] | null;
@@ -13,34 +10,13 @@ type TranscriptTopics = {
};
const useTopics = (id: string): TranscriptTopics => {
const [topics, setTopics] = useState<Topic[] | null>(null);
const [loading, setLoading] = useState<boolean>(true);
const [error, setErrorState] = useState<Error | null>(null);
const { setError } = useError();
const api = useApi();
useEffect(() => {
if (!id || !api) return;
const { data: topics, isLoading: loading, error } = useTranscriptTopics(id);
setLoading(true);
api
.v1TranscriptGetTopics({ transcriptId: id })
.then((result) => {
setTopics(result);
setLoading(false);
console.debug("Transcript topics loaded:", result);
})
.catch((err) => {
setErrorState(err);
const shouldShowHuman = shouldShowError(err);
if (shouldShowHuman) {
setError(err, "There was an error loading the topics");
} else {
setError(err);
}
});
}, [id, !api]);
return { topics, loading, error };
return {
topics: topics || null,
loading,
error: error as Error | null,
};
};
export default useTopics;

70
www/app/(app)/transcripts/useTranscript.ts
View File

@@ -1,70 +0,0 @@
import { useEffect, useState } from "react";
import { GetTranscript } from "../../api";
import { useError } from "../../(errors)/errorContext";
import { shouldShowError } from "../../lib/errorUtils";
import useApi from "../../lib/useApi";
type ErrorTranscript = {
error: Error;
loading: false;
response: null;
reload: () => void;
};
type LoadingTranscript = {
response: null;
loading: true;
error: false;
reload: () => void;
};
type SuccessTranscript = {
response: GetTranscript;
loading: false;
error: null;
reload: () => void;
};
const useTranscript = (
id: string | null,
): ErrorTranscript | LoadingTranscript | SuccessTranscript => {
const [response, setResponse] = useState<GetTranscript | null>(null);
const [loading, setLoading] = useState<boolean>(true);
const [error, setErrorState] = useState<Error | null>(null);
const [reload, setReload] = useState(0);
const { setError } = useError();
const api = useApi();
const reloadHandler = () => setReload((prev) => prev + 1);
useEffect(() => {
if (!id || !api) return;
if (!response) {
setLoading(true);
}
api
.v1TranscriptGet({ transcriptId: id })
.then((result) => {
setResponse(result);
setLoading(false);
console.debug("Transcript Loaded:", result);
})
.catch((error) => {
const shouldShowHuman = shouldShowError(error);
if (shouldShowHuman) {
setError(error, "There was an error loading the transcript");
} else {
setError(error);
}
setErrorState(error);
});
}, [id, !api, reload]);
return { response, loading, error, reload: reloadHandler } as
| ErrorTranscript
| LoadingTranscript
| SuccessTranscript;
};
export default useTranscript;

47
www/app/(app)/transcripts/useWaveform.ts
View File

@@ -1,8 +1,7 @@
import { useEffect, useState } from "react";
import { AudioWaveform } from "../../api";
import { useError } from "../../(errors)/errorContext";
import useApi from "../../lib/useApi";
import { shouldShowError } from "../../lib/errorUtils";
import type { components } from "../../reflector-api";
import { useTranscriptWaveform } from "../../lib/apiHooks";
type AudioWaveform = components["schemas"]["AudioWaveform"];
type AudioWaveFormResponse = {
waveform: AudioWaveform | null;
@@ -11,35 +10,17 @@ type AudioWaveFormResponse = {
};
const useWaveform = (id: string, skip: boolean): AudioWaveFormResponse => {
const [waveform, setWaveform] = useState<AudioWaveform | null>(null);
const [loading, setLoading] = useState<boolean>(false);
const [error, setErrorState] = useState<Error | null>(null);
const { setError } = useError();
const api = useApi();
const {
data: waveform,
isLoading: loading,
error,
} = useTranscriptWaveform(skip ? null : id);
useEffect(() => {
if (!id || !api || skip) {
setLoading(false);
setErrorState(null);
setWaveform(null);
return;
}
setLoading(true);
setErrorState(null);
api
.v1TranscriptGetAudioWaveform({ transcriptId: id })
.then((result) => {
setWaveform(result);
setLoading(false);
console.debug("Transcript waveform loaded:", result);
})
.catch((err) => {
setErrorState(err);
setLoading(false);
});
}, [id, api, skip]);
return { waveform, loading, error };
return {
waveform: waveform || null,
loading,
error: error as Error | null,
};
};
export default useWaveform;

44
www/app/(app)/transcripts/useWebRTC.ts
View File

@@ -1,8 +1,9 @@
import { useEffect, useState } from "react";
import Peer from "simple-peer";
import { useError } from "../../(errors)/errorContext";
import useApi from "../../lib/useApi";
import { RtcOffer } from "../../api";
import { useTranscriptWebRTC } from "../../lib/apiHooks";
import type { components } from "../../reflector-api";
type RtcOffer = components["schemas"]["RtcOffer"];
const useWebRTC = (
stream: MediaStream | null,
@@ -10,10 +11,10 @@ const useWebRTC = (
): Peer => {
const [peer, setPeer] = useState<Peer | null>(null);
const { setError } = useError();
const api = useApi();
const { mutateAsync: mutateWebRtcTranscriptAsync } = useTranscriptWebRTC();
useEffect(() => {
if (!stream || !transcriptId || !api) {
if (!stream || !transcriptId) {
return;
}
@@ -24,7 +25,7 @@ const useWebRTC = (
try {
p = new Peer({ initiator: true, stream: stream });
} catch (error) {
setError(error, "Error creating WebRTC");
setError(error as Error, "Error creating WebRTC");
return;
}
@@ -32,26 +33,31 @@ const useWebRTC = (
setError(new Error(`WebRTC error: ${err}`));
});
p.on("signal", (data: any) => {
if (!api) return;
p.on("signal", async (data: any) => {
if ("sdp" in data) {
const rtcOffer: RtcOffer = {
sdp: data.sdp,
type: data.type,
};
api
.v1TranscriptRecordWebrtc({ transcriptId, requestBody: rtcOffer })
.then((answer) => {
try {
p.signal(answer);
} catch (error) {
setError(error);
}
})
.catch((error) => {
setError(error, "Error loading WebRTCOffer");
try {
const answer = await mutateWebRtcTranscriptAsync({
params: {
path: {
transcript_id: transcriptId,
},
},
body: rtcOffer,
});
try {
p.signal(answer);
} catch (error) {
setError(error as Error);
}
} catch (error) {
setError(error as Error, "Error loading WebRTCOffer");
}
}
});
@@ -63,7 +69,7 @@ const useWebRTC = (
return () => {
p.destroy();
};
}, [stream, transcriptId, !api]);
}, [stream, transcriptId, mutateWebRtcTranscriptAsync]);
return peer;
};

68
www/app/(app)/transcripts/useWebSockets.ts
View File

@@ -1,9 +1,12 @@
import { useContext, useEffect, useState } from "react";
import { useEffect, useState } from "react";
import { Topic, FinalSummary, Status } from "./webSocketTypes";
import { useError } from "../../(errors)/errorContext";
import { DomainContext } from "../../domainContext";
import { AudioWaveform, GetTranscriptSegmentTopic } from "../../api";
import useApi from "../../lib/useApi";
import type { components } from "../../reflector-api";
type AudioWaveform = components["schemas"]["AudioWaveform"];
type GetTranscriptSegmentTopic =
components["schemas"]["GetTranscriptSegmentTopic"];
import { useQueryClient } from "@tanstack/react-query";
import { $api, WEBSOCKET_URL } from "../../lib/apiClient";
export type UseWebSockets = {
transcriptTextLive: string;
@@ -12,7 +15,7 @@ export type UseWebSockets = {
title: string;
topics: Topic[];
finalSummary: FinalSummary;
status: Status;
status: Status | null;
waveform: AudioWaveform | null;
duration: number | null;
};
@@ -30,11 +33,10 @@ export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
const [finalSummary, setFinalSummary] = useState<FinalSummary>({
summary: "",
});
const [status, setStatus] = useState<Status>({ value: "" });
const [status, setStatus] = useState<Status | null>(null);
const { setError } = useError();
const { websocket_url } = useContext(DomainContext);
const api = useApi();
const queryClient = useQueryClient();
const [accumulatedText, setAccumulatedText] = useState<string>("");
@@ -105,6 +107,13 @@ export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
title: "Topic 1: Introduction to Quantum Mechanics",
transcript:
"A brief overview of quantum mechanics and its principles.",
segments: [
{
speaker: 1,
start: 0,
text: "This is the transcription of an example title",
},
],
},
{
id: "2",
@@ -315,11 +324,9 @@ export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
}
};
if (!transcriptId || !api) return;
if (!transcriptId) return;
api?.v1TranscriptGetWebsocketEvents({ transcriptId }).then((result) => {});
const url = `${websocket_url}/v1/transcripts/${transcriptId}/events`;
const url = `${WEBSOCKET_URL}/v1/transcripts/${transcriptId}/events`;
let ws = new WebSocket(url);
ws.onopen = () => {
@@ -361,6 +368,16 @@ export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
return [...prevTopics, topic];
});
console.debug("TOPIC event:", message.data);
// Invalidate topics query to sync with WebSocket data
queryClient.invalidateQueries({
queryKey: $api.queryOptions(
"get",
"/v1/transcripts/{transcript_id}/topics",
{
params: { path: { transcript_id: transcriptId } },
},
).queryKey,
});
break;
case "FINAL_SHORT_SUMMARY":
@@ -370,6 +387,16 @@ export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
case "FINAL_LONG_SUMMARY":
if (message.data) {
setFinalSummary(message.data);
// Invalidate transcript query to sync summary
queryClient.invalidateQueries({
queryKey: $api.queryOptions(
"get",
"/v1/transcripts/{transcript_id}",
{
params: { path: { transcript_id: transcriptId } },
},
).queryKey,
});
}
break;
@@ -377,6 +404,16 @@ export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
console.debug("FINAL_TITLE event:", message.data);
if (message.data) {
setTitle(message.data.title);
// Invalidate transcript query to sync title
queryClient.invalidateQueries({
queryKey: $api.queryOptions(
"get",
"/v1/transcripts/{transcript_id}",
{
params: { path: { transcript_id: transcriptId } },
},
).queryKey,
});
}
break;
@@ -434,6 +471,11 @@ export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
break;
case 1001: // Navigate away
break;
case 1006: // Closed by client Chrome
console.warn(
"WebSocket closed by client, likely duplicated connection in react dev mode",
);
break;
default:
setError(
new Error(`WebSocket closed unexpectedly with code: ${event.code}`),
@@ -450,7 +492,7 @@ export const useWebSockets = (transcriptId: string | null): UseWebSockets => {
return () => {
ws.close();
};
}, [transcriptId, !api]);
}, [transcriptId]);
return {
transcriptTextLive,

7
www/app/(app)/transcripts/webSocketTypes.ts
View File

@@ -1,4 +1,7 @@
import { GetTranscriptTopic } from "../../api";
import type { components } from "../../reflector-api";
import type { TranscriptStatus } from "../../lib/transcript";
type GetTranscriptTopic = components["schemas"]["GetTranscriptTopic"];
export type Topic = GetTranscriptTopic;
@@ -11,7 +14,7 @@ export type FinalSummary = {
};
export type Status = {
value: string;
value: TranscriptStatus;
};
export type TranslatedTopic = {

17
www/app/(auth)/userInfo.tsx
View File

@@ -1,18 +1,21 @@
"use client";
import { signOut, signIn } from "next-auth/react";
import useSessionStatus from "../lib/useSessionStatus";
import { Spinner, Link } from "@chakra-ui/react";
import { useAuth } from "../lib/AuthProvider";
export default function UserInfo() {
const { isLoading, isAuthenticated } = useSessionStatus();
const auth = useAuth();
const status = auth.status;
const isLoading = status === "loading";
const isAuthenticated = status === "authenticated";
const isRefreshing = status === "refreshing";
return isLoading ? (
<Spinner size="xs" className="mx-3" />
) : !isAuthenticated ? (
) : !isAuthenticated && !isRefreshing ? (
<Link
href="/"
className="font-light px-2"
onClick={() => signIn("authentik")}
onClick={() => auth.signIn("authentik")}
>
Log in
</Link>
@@ -20,7 +23,7 @@ export default function UserInfo() {
<Link
href="#"
className="font-light px-2"
onClick={() => signOut({ callbackUrl: "/" })}
onClick={() => auth.signOut({ callbackUrl: "/" })}
>
Log out
</Link>

15
www/app/[roomName]/page.tsx
View File

@@ -1,25 +1,28 @@
"use client";
import { useCallback, useEffect, useState } from "react";
import { useCallback, useEffect, useState, use } from "react";
import { Box, Spinner } from "@chakra-ui/react";
import useRoomMeeting from "./useRoomMeeting";
import { useRouter } from "next/navigation";
import { notFound } from "next/navigation";
import useSessionStatus from "../lib/useSessionStatus";
import { useAuth } from "../lib/AuthProvider";
import VideoPlatformEmbed from "../lib/videoPlatforms/VideoPlatformEmbed";
export type RoomDetails = {
params: {
params: Promise<{
roomName: string;
};
}>;
};
export default function Room(details: RoomDetails) {
const [platformReady, setPlatformReady] = useState(false);
const roomName = details.params.roomName;
const params = use(details.params);
const roomName = params.roomName;
const meeting = useRoomMeeting(roomName);
const router = useRouter();
const { isLoading, isAuthenticated } = useSessionStatus();
const status = useAuth().status;
const isAuthenticated = status === "authenticated";
const isLoading = status === "loading" || meeting.loading;
const handleLeave = useCallback(() => {
router.push("/browse");

43
www/app/[roomName]/useRoomMeeting.tsx
View File

@@ -1,8 +1,10 @@
import { useEffect, useState } from "react";
import { useError } from "../(errors)/errorContext";
import { Meeting } from "../api";
import type { components } from "../reflector-api";
import { shouldShowError } from "../lib/errorUtils";
import useApi from "../lib/useApi";
type Meeting = components["schemas"]["Meeting"];
import { useRoomsCreateMeeting } from "../lib/apiHooks";
import { notFound } from "next/navigation";
type ErrorMeeting = {
@@ -30,27 +32,25 @@ const useRoomMeeting = (
roomName: string | null | undefined,
): ErrorMeeting | LoadingMeeting | SuccessMeeting => {
const [response, setResponse] = useState<Meeting | null>(null);
const [loading, setLoading] = useState<boolean>(true);
const [error, setErrorState] = useState<Error | null>(null);
const [reload, setReload] = useState(0);
const { setError } = useError();
const api = useApi();
const createMeetingMutation = useRoomsCreateMeeting();
const reloadHandler = () => setReload((prev) => prev + 1);
useEffect(() => {
if (!roomName || !api) return;
if (!roomName) return;
if (!response) {
setLoading(true);
}
api
.v1RoomsCreateMeeting({ roomName })
.then((result) => {
const createMeeting = async () => {
try {
const result = await createMeetingMutation.mutateAsync({
params: {
path: {
room_name: roomName,
},
},
});
setResponse(result);
setLoading(false);
})
.catch((error) => {
} catch (error: any) {
const shouldShowHuman = shouldShowError(error);
if (shouldShowHuman && error.status !== 404) {
setError(
@@ -60,9 +60,14 @@ const useRoomMeeting = (
} else {
setError(error);
}
setErrorState(error);
});
}, [roomName, !api, reload]);
}
};
createMeeting();
}, [roomName, reload]);
const loading = createMeetingMutation.isPending && !response;
const error = createMeetingMutation.error as Error | null;
return { response, loading, error, reload: reloadHandler } as
| ErrorMeeting

37
www/app/api/OpenApi.ts
View File

@@ -1,37 +0,0 @@
import type { BaseHttpRequest } from "./core/BaseHttpRequest";
import type { OpenAPIConfig } from "./core/OpenAPI";
import { Interceptors } from "./core/OpenAPI";
import { AxiosHttpRequest } from "./core/AxiosHttpRequest";
import { DefaultService } from "./services.gen";
type HttpRequestConstructor = new (config: OpenAPIConfig) => BaseHttpRequest;
export class OpenApi {
public readonly default: DefaultService;
public readonly request: BaseHttpRequest;
constructor(
config?: Partial<OpenAPIConfig>,
HttpRequest: HttpRequestConstructor = AxiosHttpRequest,
) {
this.request = new HttpRequest({
BASE: config?.BASE ?? "",
VERSION: config?.VERSION ?? "0.1.0",
WITH_CREDENTIALS: config?.WITH_CREDENTIALS ?? false,
CREDENTIALS: config?.CREDENTIALS ?? "include",
TOKEN: config?.TOKEN,
USERNAME: config?.USERNAME,
PASSWORD: config?.PASSWORD,
HEADERS: config?.HEADERS,
ENCODE_PATH: config?.ENCODE_PATH,
interceptors: {
request: config?.interceptors?.request ?? new Interceptors(),
response: config?.interceptors?.response ?? new Interceptors(),
},
});
this.default = new DefaultService(this.request);
}
}

7
www/app/api/auth/[...nextauth]/route.ts
View File

@@ -1,9 +1,6 @@
// NextAuth route handler for Authentik
// Refresh rotation has been taken from https://next-auth.js.org/v3/tutorials/refresh-token-rotation even if we are using 4.x
import NextAuth from "next-auth";
import { authOptions } from "../../../lib/auth";
import { authOptions } from "../../../lib/authBackend";
const handler = NextAuth(authOptions);
const handler = NextAuth(authOptions());
export { handler as GET, handler as POST };

25
www/app/api/core/ApiError.ts
View File

@@ -1,25 +0,0 @@
import type { ApiRequestOptions } from "./ApiRequestOptions";
import type { ApiResult } from "./ApiResult";
export class ApiError extends Error {
public readonly url: string;
public readonly status: number;
public readonly statusText: string;
public readonly body: unknown;
public readonly request: ApiRequestOptions;
constructor(
request: ApiRequestOptions,
response: ApiResult,
message: string,
) {
super(message);
this.name = "ApiError";
this.url = response.url;
this.status = response.status;
this.statusText = response.statusText;
this.body = response.body;
this.request = request;
}
}

21
www/app/api/core/ApiRequestOptions.ts
View File

@@ -1,21 +0,0 @@
export type ApiRequestOptions<T = unknown> = {
readonly method:
| "GET"
| "PUT"
| "POST"
| "DELETE"
| "OPTIONS"
| "HEAD"
| "PATCH";
readonly url: string;
readonly path?: Record<string, unknown>;
readonly cookies?: Record<string, unknown>;
readonly headers?: Record<string, unknown>;
readonly query?: Record<string, unknown>;
readonly formData?: Record<string, unknown>;
readonly body?: any;
readonly mediaType?: string;
readonly responseHeader?: string;
readonly responseTransformer?: (data: unknown) => Promise<T>;
readonly errors?: Record<number | string, string>;
};

7
www/app/api/core/ApiResult.ts
View File

@@ -1,7 +0,0 @@
export type ApiResult<TData = any> = {
readonly body: TData;
readonly ok: boolean;
readonly status: number;
readonly statusText: string;
readonly url: string;
};

23
www/app/api/core/AxiosHttpRequest.ts
View File

@@ -1,23 +0,0 @@
import type { ApiRequestOptions } from "./ApiRequestOptions";
import { BaseHttpRequest } from "./BaseHttpRequest";
import type { CancelablePromise } from "./CancelablePromise";
import type { OpenAPIConfig } from "./OpenAPI";
import { request as __request } from "./request";
export class AxiosHttpRequest extends BaseHttpRequest {
constructor(config: OpenAPIConfig) {
super(config);
}
/**
* Request method
* @param options The request options from the service
* @returns CancelablePromise<T>
* @throws ApiError
*/
public override request<T>(
options: ApiRequestOptions<T>,
): CancelablePromise<T> {
return __request(this.config, options);
}
}

11
www/app/api/core/BaseHttpRequest.ts
View File

@@ -1,11 +0,0 @@
import type { ApiRequestOptions } from "./ApiRequestOptions";
import type { CancelablePromise } from "./CancelablePromise";
import type { OpenAPIConfig } from "./OpenAPI";
export abstract class BaseHttpRequest {
constructor(public readonly config: OpenAPIConfig) {}
public abstract request<T>(
options: ApiRequestOptions<T>,
): CancelablePromise<T>;
}

126
www/app/api/core/CancelablePromise.ts
View File

@@ -1,126 +0,0 @@
export class CancelError extends Error {
constructor(message: string) {
super(message);
this.name = "CancelError";
}
public get isCancelled(): boolean {
return true;
}
}
export interface OnCancel {
readonly isResolved: boolean;
readonly isRejected: boolean;
readonly isCancelled: boolean;
(cancelHandler: () => void): void;
}
export class CancelablePromise<T> implements Promise<T> {
private _isResolved: boolean;
private _isRejected: boolean;
private _isCancelled: boolean;
readonly cancelHandlers: (() => void)[];
readonly promise: Promise<T>;
private _resolve?: (value: T | PromiseLike<T>) => void;
private _reject?: (reason?: unknown) => void;
constructor(
executor: (
resolve: (value: T | PromiseLike<T>) => void,
reject: (reason?: unknown) => void,
onCancel: OnCancel,
) => void,
) {
this._isResolved = false;
this._isRejected = false;
this._isCancelled = false;
this.cancelHandlers = [];
this.promise = new Promise<T>((resolve, reject) => {
this._resolve = resolve;
this._reject = reject;
const onResolve = (value: T | PromiseLike<T>): void => {
if (this._isResolved || this._isRejected || this._isCancelled) {
return;
}
this._isResolved = true;
if (this._resolve) this._resolve(value);
};
const onReject = (reason?: unknown): void => {
if (this._isResolved || this._isRejected || this._isCancelled) {
return;
}
this._isRejected = true;
if (this._reject) this._reject(reason);
};
const onCancel = (cancelHandler: () => void): void => {
if (this._isResolved || this._isRejected || this._isCancelled) {
return;
}
this.cancelHandlers.push(cancelHandler);
};
Object.defineProperty(onCancel, "isResolved", {
get: (): boolean => this._isResolved,
});
Object.defineProperty(onCancel, "isRejected", {
get: (): boolean => this._isRejected,
});
Object.defineProperty(onCancel, "isCancelled", {
get: (): boolean => this._isCancelled,
});
return executor(onResolve, onReject, onCancel as OnCancel);
});
}
get [Symbol.toStringTag]() {
return "Cancellable Promise";
}
public then<TResult1 = T, TResult2 = never>(
onFulfilled?: ((value: T) => TResult1 | PromiseLike<TResult1>) | null,
onRejected?: ((reason: unknown) => TResult2 | PromiseLike<TResult2>) | null,
): Promise<TResult1 | TResult2> {
return this.promise.then(onFulfilled, onRejected);
}
public catch<TResult = never>(
onRejected?: ((reason: unknown) => TResult | PromiseLike<TResult>) | null,
): Promise<T | TResult> {
return this.promise.catch(onRejected);
}
public finally(onFinally?: (() => void) | null): Promise<T> {
return this.promise.finally(onFinally);
}
public cancel(): void {
if (this._isResolved || this._isRejected || this._isCancelled) {
return;
}
this._isCancelled = true;
if (this.cancelHandlers.length) {
try {
for (const cancelHandler of this.cancelHandlers) {
cancelHandler();
}
} catch (error) {
console.warn("Cancellation threw an error", error);
return;
}
}
this.cancelHandlers.length = 0;
if (this._reject) this._reject(new CancelError("Request aborted"));
}
public get isCancelled(): boolean {
return this._isCancelled;
}
}

57
www/app/api/core/OpenAPI.ts
View File

@@ -1,57 +0,0 @@
import type { AxiosRequestConfig, AxiosResponse } from "axios";
import type { ApiRequestOptions } from "./ApiRequestOptions";
type Headers = Record<string, string>;
type Middleware<T> = (value: T) => T | Promise<T>;
type Resolver<T> = (options: ApiRequestOptions<T>) => Promise<T>;
export class Interceptors<T> {
_fns: Middleware<T>[];
constructor() {
this._fns = [];
}
eject(fn: Middleware<T>): void {
const index = this._fns.indexOf(fn);
if (index !== -1) {
this._fns = [...this._fns.slice(0, index), ...this._fns.slice(index + 1)];
}
}
use(fn: Middleware<T>): void {
this._fns = [...this._fns, fn];
}
}
export type OpenAPIConfig = {
BASE: string;
CREDENTIALS: "include" | "omit" | "same-origin";
ENCODE_PATH?: ((path: string) => string) | undefined;
HEADERS?: Headers | Resolver<Headers> | undefined;
PASSWORD?: string | Resolver<string> | undefined;
TOKEN?: string | Resolver<string> | undefined;
USERNAME?: string | Resolver<string> | undefined;
VERSION: string;
WITH_CREDENTIALS: boolean;
interceptors: {
request: Interceptors<AxiosRequestConfig>;
response: Interceptors<AxiosResponse>;
};
};
export const OpenAPI: OpenAPIConfig = {
BASE: "",
CREDENTIALS: "include",
ENCODE_PATH: undefined,
HEADERS: undefined,
PASSWORD: undefined,
TOKEN: undefined,
USERNAME: undefined,
VERSION: "0.1.0",
WITH_CREDENTIALS: false,
interceptors: {
request: new Interceptors(),
response: new Interceptors(),
},
};

387
www/app/api/core/request.ts
View File

@@ -1,387 +0,0 @@
import axios from "axios";
import type {
AxiosError,
AxiosRequestConfig,
AxiosResponse,
AxiosInstance,
} from "axios";
import { ApiError } from "./ApiError";
import type { ApiRequestOptions } from "./ApiRequestOptions";
import type { ApiResult } from "./ApiResult";
import { CancelablePromise } from "./CancelablePromise";
import type { OnCancel } from "./CancelablePromise";
import type { OpenAPIConfig } from "./OpenAPI";
export const isString = (value: unknown): value is string => {
return typeof value === "string";
};
export const isStringWithValue = (value: unknown): value is string => {
return isString(value) && value !== "";
};
export const isBlob = (value: any): value is Blob => {
return value instanceof Blob;
};
export const isFormData = (value: unknown): value is FormData => {
return value instanceof FormData;
};
export const isSuccess = (status: number): boolean => {
return status >= 200 && status < 300;
};
export const base64 = (str: string): string => {
try {
return btoa(str);
} catch (err) {
// @ts-ignore
return Buffer.from(str).toString("base64");
}
};
export const getQueryString = (params: Record<string, unknown>): string => {
const qs: string[] = [];
const append = (key: string, value: unknown) => {
qs.push(`${encodeURIComponent(key)}=${encodeURIComponent(String(value))}`);
};
const encodePair = (key: string, value: unknown) => {
if (value === undefined || value === null) {
return;
}
if (value instanceof Date) {
append(key, value.toISOString());
} else if (Array.isArray(value)) {
value.forEach((v) => encodePair(key, v));
} else if (typeof value === "object") {
Object.entries(value).forEach(([k, v]) => encodePair(`${key}[${k}]`, v));
} else {
append(key, value);
}
};
Object.entries(params).forEach(([key, value]) => encodePair(key, value));
return qs.length ? `?${qs.join("&")}` : "";
};
const getUrl = (config: OpenAPIConfig, options: ApiRequestOptions): string => {
const encoder = config.ENCODE_PATH || encodeURI;
const path = options.url
.replace("{api-version}", config.VERSION)
.replace(/{(.*?)}/g, (substring: string, group: string) => {
if (options.path?.hasOwnProperty(group)) {
return encoder(String(options.path[group]));
}
return substring;
});
const url = config.BASE + path;
return options.query ? url + getQueryString(options.query) : url;
};
export const getFormData = (
options: ApiRequestOptions,
): FormData | undefined => {
if (options.formData) {
const formData = new FormData();
const process = (key: string, value: unknown) => {
if (isString(value) || isBlob(value)) {
formData.append(key, value);
} else {
formData.append(key, JSON.stringify(value));
}
};
Object.entries(options.formData)
.filter(([, value]) => value !== undefined && value !== null)
.forEach(([key, value]) => {
if (Array.isArray(value)) {
value.forEach((v) => process(key, v));
} else {
process(key, value);
}
});
return formData;
}
return undefined;
};
type Resolver<T> = (options: ApiRequestOptions<T>) => Promise<T>;
export const resolve = async <T>(
options: ApiRequestOptions<T>,
resolver?: T | Resolver<T>,
): Promise<T | undefined> => {
if (typeof resolver === "function") {
return (resolver as Resolver<T>)(options);
}
return resolver;
};
export const getHeaders = async <T>(
config: OpenAPIConfig,
options: ApiRequestOptions<T>,
): Promise<Record<string, string>> => {
const [token, username, password, additionalHeaders] = await Promise.all([
// @ts-ignore
resolve(options, config.TOKEN),
// @ts-ignore
resolve(options, config.USERNAME),
// @ts-ignore
resolve(options, config.PASSWORD),
// @ts-ignore
resolve(options, config.HEADERS),
]);
const headers = Object.entries({
Accept: "application/json",
...additionalHeaders,
...options.headers,
})
.filter(([, value]) => value !== undefined && value !== null)
.reduce(
(headers, [key, value]) => ({
...headers,
[key]: String(value),
}),
{} as Record<string, string>,
);
if (isStringWithValue(token)) {
headers["Authorization"] = `Bearer ${token}`;
}
if (isStringWithValue(username) && isStringWithValue(password)) {
const credentials = base64(`${username}:${password}`);
headers["Authorization"] = `Basic ${credentials}`;
}
if (options.body !== undefined) {
if (options.mediaType) {
headers["Content-Type"] = options.mediaType;
} else if (isBlob(options.body)) {
headers["Content-Type"] = options.body.type || "application/octet-stream";
} else if (isString(options.body)) {
headers["Content-Type"] = "text/plain";
} else if (!isFormData(options.body)) {
headers["Content-Type"] = "application/json";
}
} else if (options.formData !== undefined) {
if (options.mediaType) {
headers["Content-Type"] = options.mediaType;
}
}
return headers;
};
export const getRequestBody = (options: ApiRequestOptions): unknown => {
if (options.body) {
return options.body;
}
return undefined;
};
export const sendRequest = async <T>(
config: OpenAPIConfig,
options: ApiRequestOptions<T>,
url: string,
body: unknown,
formData: FormData | undefined,
headers: Record<string, string>,
onCancel: OnCancel,
axiosClient: AxiosInstance,
): Promise<AxiosResponse<T>> => {
const controller = new AbortController();
let requestConfig: AxiosRequestConfig = {
data: body ?? formData,
headers,
method: options.method,
signal: controller.signal,
url,
withCredentials: config.WITH_CREDENTIALS,
};
onCancel(() => controller.abort());
for (const fn of config.interceptors.request._fns) {
requestConfig = await fn(requestConfig);
}
try {
return await axiosClient.request(requestConfig);
} catch (error) {
const axiosError = error as AxiosError<T>;
if (axiosError.response) {
return axiosError.response;
}
throw error;
}
};
export const getResponseHeader = (
response: AxiosResponse<unknown>,
responseHeader?: string,
): string | undefined => {
if (responseHeader) {
const content = response.headers[responseHeader];
if (isString(content)) {
return content;
}
}
return undefined;
};
export const getResponseBody = (response: AxiosResponse<unknown>): unknown => {
if (response.status !== 204) {
return response.data;
}
return undefined;
};
export const catchErrorCodes = (
options: ApiRequestOptions,
result: ApiResult,
): void => {
const errors: Record<number, string> = {
400: "Bad Request",
401: "Unauthorized",
402: "Payment Required",
403: "Forbidden",
404: "Not Found",
405: "Method Not Allowed",
406: "Not Acceptable",
407: "Proxy Authentication Required",
408: "Request Timeout",
409: "Conflict",
410: "Gone",
411: "Length Required",
412: "Precondition Failed",
413: "Payload Too Large",
414: "URI Too Long",
415: "Unsupported Media Type",
416: "Range Not Satisfiable",
417: "Expectation Failed",
418: "Im a teapot",
421: "Misdirected Request",
422: "Unprocessable Content",
423: "Locked",
424: "Failed Dependency",
425: "Too Early",
426: "Upgrade Required",
428: "Precondition Required",
429: "Too Many Requests",
431: "Request Header Fields Too Large",
451: "Unavailable For Legal Reasons",
500: "Internal Server Error",
501: "Not Implemented",
502: "Bad Gateway",
503: "Service Unavailable",
504: "Gateway Timeout",
505: "HTTP Version Not Supported",
506: "Variant Also Negotiates",
507: "Insufficient Storage",
508: "Loop Detected",
510: "Not Extended",
511: "Network Authentication Required",
...options.errors,
};
const error = errors[result.status];
if (error) {
throw new ApiError(options, result, error);
}
if (!result.ok) {
const errorStatus = result.status ?? "unknown";
const errorStatusText = result.statusText ?? "unknown";
const errorBody = (() => {
try {
return JSON.stringify(result.body, null, 2);
} catch (e) {
return undefined;
}
})();
throw new ApiError(
options,
result,
`Generic Error: status: ${errorStatus}; status text: ${errorStatusText}; body: ${errorBody}`,
);
}
};
/**
* Request method
* @param config The OpenAPI configuration object
* @param options The request options from the service
* @param axiosClient The axios client instance to use
* @returns CancelablePromise<T>
* @throws ApiError
*/
export const request = <T>(
config: OpenAPIConfig,
options: ApiRequestOptions<T>,
axiosClient: AxiosInstance = axios,
): CancelablePromise<T> => {
return new CancelablePromise(async (resolve, reject, onCancel) => {
try {
const url = getUrl(config, options);
const formData = getFormData(options);
const body = getRequestBody(options);
const headers = await getHeaders(config, options);
if (!onCancel.isCancelled) {
let response = await sendRequest<T>(
config,
options,
url,
body,
formData,
headers,
onCancel,
axiosClient,
);
for (const fn of config.interceptors.response._fns) {
response = await fn(response);
}
const responseBody = getResponseBody(response);
const responseHeader = getResponseHeader(
response,
options.responseHeader,
);
let transformedBody = responseBody;
if (options.responseTransformer && isSuccess(response.status)) {
transformedBody = await options.responseTransformer(responseBody);
}
const result: ApiResult = {
url,
ok: isSuccess(response.status),
status: response.status,
statusText: response.statusText,
body: responseHeader ?? transformedBody,
};
catchErrorCodes(options, result);
resolve(result.body);
}
} catch (error) {
reject(error);
}
});
};

9
www/app/api/index.ts
View File

@@ -1,9 +0,0 @@
// This file is auto-generated by @hey-api/openapi-ts
export { OpenApi } from "./OpenApi";
export { ApiError } from "./core/ApiError";
export { BaseHttpRequest } from "./core/BaseHttpRequest";
export { CancelablePromise, CancelError } from "./core/CancelablePromise";
export { OpenAPI, type OpenAPIConfig } from "./core/OpenAPI";
export * from "./schemas.gen";
export * from "./services.gen";
export * from "./types.gen";

1840
www/app/api/schemas.gen.ts
View File

File diff suppressed because it is too large Load Diff

1012
www/app/api/services.gen.ts
View File

File diff suppressed because it is too large Load Diff

1218
www/app/api/types.gen.ts
View File

File diff suppressed because it is too large Load Diff

1
www/app/api/urls.ts
View File

@@ -1,2 +1 @@
// TODO better connection with generated schema; it's duplication
export const RECORD_A_MEETING_URL = "/transcripts/new" as const;

49
www/app/domainContext.tsx
View File

@@ -1,49 +0,0 @@
"use client";
import { createContext, useContext, useEffect, useState } from "react";
import { DomainConfig } from "./lib/edgeConfig";
type DomainContextType = Omit<DomainConfig, "auth_callback_url">;
export const DomainContext = createContext<DomainContextType>({
features: {
requireLogin: false,
privacy: true,
browse: false,
sendToZulip: false,
},
api_url: "",
websocket_url: "",
});
export const DomainContextProvider = ({
config,
children,
}: {
config: DomainConfig;
children: any;
}) => {
const [context, setContext] = useState<DomainContextType>();
useEffect(() => {
if (!config) return;
const { auth_callback_url, ...others } = config;
setContext(others);
}, [config]);
if (!context) return;
return (
<DomainContext.Provider value={context}>{children}</DomainContext.Provider>
);
};
// Get feature config client-side with
export const featureEnabled = (
featureName: "requireLogin" | "privacy" | "browse" | "sendToZulip",
) => {
const context = useContext(DomainContext);
return context.features[featureName] as boolean | undefined;
};
// Get config server-side (out of react) : see lib/edgeConfig.

33
www/app/layout.tsx
View File

@@ -1,14 +1,12 @@
import "./styles/globals.scss";
import { Metadata, Viewport } from "next";
import { Poppins } from "next/font/google";
import SessionProvider from "./lib/SessionProvider";
import { ErrorProvider } from "./(errors)/errorContext";
import ErrorMessage from "./(errors)/errorMessage";
import { DomainContextProvider } from "./domainContext";
import { RecordingConsentProvider } from "./recordingConsentContext";
import { getConfig } from "./lib/edgeConfig";
import { ErrorBoundary } from "@sentry/nextjs";
import { Providers } from "./providers";
import { assertExistsAndNonEmptyString } from "./lib/utils";
const poppins = Poppins({
subsets: ["latin"],
@@ -23,8 +21,13 @@ export const viewport: Viewport = {
maximumScale: 1,
};
const NEXT_PUBLIC_SITE_URL = assertExistsAndNonEmptyString(
process.env.NEXT_PUBLIC_SITE_URL,
"NEXT_PUBLIC_SITE_URL required",
);
export const metadata: Metadata = {
metadataBase: new URL(process.env.NEXT_PUBLIC_SITE_URL!),
metadataBase: new URL(NEXT_PUBLIC_SITE_URL),
title: {
template: "%s – Reflector",
default: "Reflector - AI-Powered Meeting Transcriptions by Monadical",
@@ -69,23 +72,17 @@ export default async function RootLayout({
}: {
children: React.ReactNode;
}) {
const config = await getConfig();
return (
<html lang="en" className={poppins.className} suppressHydrationWarning>
<body className={"h-[100svh] w-[100svw] overflow-x-hidden relative"}>
<SessionProvider>
<DomainContextProvider config={config}>
<RecordingConsentProvider>
<ErrorBoundary fallback={<p>"something went really wrong"</p>}>
<ErrorProvider>
<ErrorMessage />
<Providers>{children}</Providers>
</ErrorProvider>
</ErrorBoundary>
</RecordingConsentProvider>
</DomainContextProvider>
</SessionProvider>
<RecordingConsentProvider>
<ErrorBoundary fallback={<p>"something went really wrong"</p>}>
<ErrorProvider>
<ErrorMessage />
<Providers>{children}</Providers>
</ErrorProvider>
</ErrorBoundary>
</RecordingConsentProvider>
</body>
</html>
);

132
www/app/lib/AuthProvider.tsx Normal file
View File

@@ -0,0 +1,132 @@
"use client";
import { createContext, useContext } from "react";
import { useSession as useNextAuthSession } from "next-auth/react";
import { signOut, signIn } from "next-auth/react";
import { configureApiAuth } from "./apiClient";
import { assertCustomSession, CustomSession } from "./types";
import { Session } from "next-auth";
import { SessionAutoRefresh } from "./SessionAutoRefresh";
import { REFRESH_ACCESS_TOKEN_ERROR } from "./auth";
import { assertExists } from "./utils";
import { featureEnabled } from "./features";
type AuthContextType = (
| { status: "loading" }
| { status: "refreshing"; user: CustomSession["user"] }
| { status: "unauthenticated"; error?: string }
| {
status: "authenticated";
accessToken: string;
accessTokenExpires: number;
user: CustomSession["user"];
}
) & {
update: () => Promise<Session | null>;
signIn: typeof signIn;
signOut: typeof signOut;
};
const AuthContext = createContext<AuthContextType | undefined>(undefined);
const isAuthEnabled = featureEnabled("requireLogin");
const noopAuthContext: AuthContextType = {
status: "unauthenticated",
update: async () => {
return null;
},
signIn: async () => {
throw new Error("signIn not supposed to be called");
},
signOut: async () => {
throw new Error("signOut not supposed to be called");
},
};
export function AuthProvider({ children }: { children: React.ReactNode }) {
const { data: session, status, update } = useNextAuthSession();
const contextValue: AuthContextType = isAuthEnabled
? {
...(() => {
switch (status) {
case "loading": {
const sessionIsHere = !!session;
// actually exists sometimes; nextAuth types are something else
switch (sessionIsHere as boolean) {
case false: {
return { status };
}
case true: {
return {
status: "refreshing" as const,
user: assertCustomSession(
assertExists(session as unknown as Session),
).user,
};
}
default: {
throw new Error("unreachable");
}
}
}
case "authenticated": {
const customSession = assertCustomSession(session);
if (customSession?.error === REFRESH_ACCESS_TOKEN_ERROR) {
// token had expired but next auth still returns "authenticated" so show user unauthenticated state
return {
status: "unauthenticated" as const,
};
} else if (customSession?.accessToken) {
return {
status,
accessToken: customSession.accessToken,
accessTokenExpires: customSession.accessTokenExpires,
user: customSession.user,
};
} else {
console.warn(
"illegal state: authenticated but have no session/or access token. ignoring",
);
return { status: "unauthenticated" as const };
}
}
case "unauthenticated": {
return { status: "unauthenticated" as const };
}
default: {
const _: never = status;
throw new Error("unreachable");
}
}
})(),
update,
signIn,
signOut,
}
: noopAuthContext;
// not useEffect, we need it ASAP
// apparently, still no guarantee this code runs before mutations are fired
configureApiAuth(
contextValue.status === "authenticated"
? contextValue.accessToken
: contextValue.status === "loading"
? undefined
: null,
);
return (
<AuthContext.Provider value={contextValue}>
<SessionAutoRefresh>{children}</SessionAutoRefresh>
</AuthContext.Provider>
);
}
export function useAuth() {
const context = useContext(AuthContext);
if (context === undefined) {
throw new Error("useAuth must be used within an AuthProvider");
}
return context;
}

39
www/app/lib/SessionAutoRefresh.tsx
View File

@@ -1,5 +1,5 @@
/**
* This is a custom hook that automatically refreshes the session when the access token is about to expire.
* This is a custom provider that automatically refreshes the session when the access token is about to expire.
* When communicating with the reflector API, we need to ensure that the access token is always valid.
*
* We could have implemented that as an interceptor on the API client, but not everything is using the
@@ -7,30 +7,35 @@
*/
"use client";
import { useSession } from "next-auth/react";
import { useEffect } from "react";
import { CustomSession } from "./types";
import { useAuth } from "./AuthProvider";
import { shouldRefreshToken } from "./auth";
export function SessionAutoRefresh({
children,
refreshInterval = 20 /* seconds */,
}) {
const { data: session, update } = useSession();
const customSession = session as CustomSession;
const accessTokenExpires = customSession?.accessTokenExpires;
export function SessionAutoRefresh({ children }) {
const auth = useAuth();
const accessTokenExpires =
auth.status === "authenticated" ? auth.accessTokenExpires : null;
useEffect(() => {
// technical value for how often the setInterval will be polling news - not too fast (no spam in case of errors)
// and not too slow (debuggable)
const INTERVAL_REFRESH_MS = 5000;
const interval = setInterval(() => {
if (accessTokenExpires) {
const timeLeft = accessTokenExpires - Date.now();
if (timeLeft < refreshInterval * 1000) {
update();
}
if (accessTokenExpires === null) return;
if (shouldRefreshToken(accessTokenExpires)) {
auth
.update()
.then(() => {})
.catch((e) => {
// note: 401 won't be considered error here
console.error("error refreshing auth token", e);
});
}
}, refreshInterval * 1000);
}, INTERVAL_REFRESH_MS);
return () => clearInterval(interval);
}, [accessTokenExpires, refreshInterval, update]);
}, [accessTokenExpires, auth.update]);
return children;
}

11
www/app/lib/SessionProvider.tsx
View File

@@ -1,11 +0,0 @@
"use client";
import { SessionProvider as SessionProviderNextAuth } from "next-auth/react";
import { SessionAutoRefresh } from "./SessionAutoRefresh";
export default function SessionProvider({ children }) {
return (
<SessionProviderNextAuth>
<SessionAutoRefresh>{children}</SessionAutoRefresh>
</SessionProviderNextAuth>
);
}

85
www/app/lib/__tests__/redisTokenCache.test.ts Normal file
View File

@@ -0,0 +1,85 @@
import {
getTokenCache,
setTokenCache,
deleteTokenCache,
TokenCacheEntry,
KV,
} from "../redisTokenCache";
const mockKV: KV & {
clear: () => void;
} = (() => {
const data = new Map<string, string>();
return {
async get(key: string): Promise<string | null> {
return data.get(key) || null;
},
async setex(key: string, seconds_: number, value: string): Promise<"OK"> {
data.set(key, value);
return "OK";
},
async del(key: string): Promise<number> {
const existed = data.has(key);
data.delete(key);
return existed ? 1 : 0;
},
clear() {
data.clear();
},
};
})();
describe("Redis Token Cache", () => {
beforeEach(() => {
mockKV.clear();
});
test("basic write/read - value written equals value read", async () => {
const testKey = "token:test-user-123";
const testValue: TokenCacheEntry = {
token: {
sub: "test-user-123",
name: "Test User",
email: "test@example.com",
accessToken: "access-token-123",
accessTokenExpires: Date.now() + 3600000, // 1 hour from now
refreshToken: "refresh-token-456",
},
timestamp: Date.now(),
};
await setTokenCache(mockKV, testKey, testValue);
const retrievedValue = await getTokenCache(mockKV, testKey);
expect(retrievedValue).not.toBeNull();
expect(retrievedValue).toEqual(testValue);
expect(retrievedValue?.token.accessToken).toBe(testValue.token.accessToken);
expect(retrievedValue?.token.sub).toBe(testValue.token.sub);
expect(retrievedValue?.timestamp).toBe(testValue.timestamp);
});
test("get returns null for non-existent key", async () => {
const result = await getTokenCache(mockKV, "non-existent-key");
expect(result).toBeNull();
});
test("delete removes token from cache", async () => {
const testKey = "token:delete-test";
const testValue: TokenCacheEntry = {
token: {
accessToken: "test-token",
accessTokenExpires: Date.now() + 3600000,
},
timestamp: Date.now(),
};
await setTokenCache(mockKV, testKey, testValue);
await deleteTokenCache(mockKV, testKey);
const result = await getTokenCache(mockKV, testKey);
expect(result).toBeNull();
});
});

72
www/app/lib/apiClient.tsx Normal file
View File

@@ -0,0 +1,72 @@
"use client";
import createClient from "openapi-fetch";
import type { paths } from "../reflector-api";
import createFetchClient from "openapi-react-query";
import { assertExistsAndNonEmptyString, parseNonEmptyString } from "./utils";
import { isBuildPhase } from "./next";
import { getSession } from "next-auth/react";
import { assertExtendedToken } from "./types";
export const API_URL = !isBuildPhase
? assertExistsAndNonEmptyString(
process.env.NEXT_PUBLIC_API_URL,
"NEXT_PUBLIC_API_URL required",
)
: "http://localhost";
// TODO decide strict validation or not
export const WEBSOCKET_URL =
process.env.NEXT_PUBLIC_WEBSOCKET_URL || "ws://127.0.0.1:1250";
export const client = createClient<paths>({
baseUrl: API_URL,
});
// will assert presence/absence of login initially
const initialSessionPromise = getSession();
const waitForAuthTokenDefinitivePresenceOrAbsence = async () => {
const initialSession = await initialSessionPromise;
if (currentAuthToken === undefined) {
currentAuthToken =
initialSession === null
? null
: assertExtendedToken(initialSession).accessToken;
}
// otherwise already overwritten by external forces
return currentAuthToken;
};
client.use({
async onRequest({ request }) {
const token = await waitForAuthTokenDefinitivePresenceOrAbsence();
if (token !== null) {
request.headers.set(
"Authorization",
`Bearer ${parseNonEmptyString(token)}`,
);
}
// XXX Only set Content-Type if not already set (FormData will set its own boundary)
// This is a work around for uploading file, we're passing a formdata
// but the content type was still application/json
if (
!request.headers.has("Content-Type") &&
!(request.body instanceof FormData)
) {
request.headers.set("Content-Type", "application/json");
}
return request;
},
});
export const $api = createFetchClient<paths>(client);
let currentAuthToken: string | null | undefined = undefined;
// the function contract: lightweight, idempotent
export const configureApiAuth = (token: string | null | undefined) => {
// watch only for the initial loading; "reloading" state assumes token presence/absence
if (token === undefined && currentAuthToken !== undefined) return;
currentAuthToken = token;
};

612
www/app/lib/apiHooks.ts Normal file
View File

@@ -0,0 +1,612 @@
"use client";
import { $api } from "./apiClient";
import { useError } from "../(errors)/errorContext";
import { useQueryClient } from "@tanstack/react-query";
import type { components } from "../reflector-api";
import { useAuth } from "./AuthProvider";
/*
* XXX error types returned from the hooks are not always correct; declared types are ValidationError but real type could be string or any other
* this is either a limitation or incorrect usage of Python json schema generator
* or, limitation or incorrect usage of .d type generator from json schema
* */
const useAuthReady = () => {
const auth = useAuth();
return {
isAuthenticated: auth.status === "authenticated",
isLoading: auth.status === "loading",
};
};
export function useRoomsList(page: number = 1) {
const { isAuthenticated } = useAuthReady();
return $api.useQuery(
"get",
"/v1/rooms",
{
params: {
query: { page },
},
},
{
enabled: isAuthenticated,
},
);
}
type SourceKind = components["schemas"]["SourceKind"];
export function useTranscriptsSearch(
q: string = "",
options: {
limit?: number;
offset?: number;
room_id?: string;
source_kind?: SourceKind;
} = {},
) {
return $api.useQuery(
"get",
"/v1/transcripts/search",
{
params: {
query: {
q,
limit: options.limit,
offset: options.offset,
room_id: options.room_id,
source_kind: options.source_kind,
},
},
},
{
enabled: true,
},
);
}
export function useTranscriptDelete() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation("delete", "/v1/transcripts/{transcript_id}", {
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: ["get", "/v1/transcripts/search"],
});
},
onError: (error) => {
setError(error as Error, "There was an error deleting the transcript");
},
});
}
export function useTranscriptProcess() {
const { setError } = useError();
return $api.useMutation("post", "/v1/transcripts/{transcript_id}/process", {
onError: (error) => {
setError(error as Error, "There was an error processing the transcript");
},
});
}
export function useTranscriptGet(transcriptId: string | null) {
return $api.useQuery(
"get",
"/v1/transcripts/{transcript_id}",
{
params: {
path: {
transcript_id: transcriptId || "",
},
},
},
{
enabled: !!transcriptId,
},
);
}
export function useRoomGet(roomId: string | null) {
const { isAuthenticated } = useAuthReady();
return $api.useQuery(
"get",
"/v1/rooms/{room_id}",
{
params: {
path: { room_id: roomId || "" },
},
},
{
enabled: !!roomId && isAuthenticated,
},
);
}
export function useRoomTestWebhook() {
const { setError } = useError();
return $api.useMutation("post", "/v1/rooms/{room_id}/webhook/test", {
onError: (error) => {
setError(error as Error, "There was an error testing the webhook");
},
});
}
export function useRoomCreate() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation("post", "/v1/rooms", {
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: $api.queryOptions("get", "/v1/rooms").queryKey,
});
},
onError: (error) => {
setError(error as Error, "There was an error creating the room");
},
});
}
export function useRoomUpdate() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation("patch", "/v1/rooms/{room_id}", {
onSuccess: async (room) => {
await Promise.all([
queryClient.invalidateQueries({
queryKey: $api.queryOptions("get", "/v1/rooms").queryKey,
}),
queryClient.invalidateQueries({
queryKey: $api.queryOptions("get", "/v1/rooms/{room_id}", {
params: {
path: {
room_id: room.id,
},
},
}).queryKey,
}),
]);
},
onError: (error) => {
setError(error as Error, "There was an error updating the room");
},
});
}
export function useRoomDelete() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation("delete", "/v1/rooms/{room_id}", {
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: $api.queryOptions("get", "/v1/rooms").queryKey,
});
},
onError: (error) => {
setError(error as Error, "There was an error deleting the room");
},
});
}
export function useZulipStreams() {
const { isAuthenticated } = useAuthReady();
return $api.useQuery(
"get",
"/v1/zulip/streams",
{},
{
enabled: isAuthenticated,
},
);
}
export function useZulipTopics(streamId: number | null) {
const { isAuthenticated } = useAuthReady();
const enabled = !!streamId && isAuthenticated;
return $api.useQuery(
"get",
"/v1/zulip/streams/{stream_id}/topics",
{
params: {
path: {
stream_id: enabled ? streamId : 0,
},
},
},
{
enabled,
},
);
}
export function useTranscriptUpdate() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation("patch", "/v1/transcripts/{transcript_id}", {
onSuccess: (data, variables) => {
queryClient.invalidateQueries({
queryKey: $api.queryOptions("get", "/v1/transcripts/{transcript_id}", {
params: {
path: { transcript_id: variables.params.path.transcript_id },
},
}).queryKey,
});
},
onError: (error) => {
setError(error as Error, "There was an error updating the transcript");
},
});
}
export function useTranscriptPostToZulip() {
const { setError } = useError();
// @ts-ignore - Zulip endpoint not in OpenAPI spec
return $api.useMutation("post", "/v1/transcripts/{transcript_id}/zulip", {
onError: (error) => {
setError(error as Error, "There was an error posting to Zulip");
},
});
}
export function useTranscriptUploadAudio() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation(
"post",
"/v1/transcripts/{transcript_id}/record/upload",
{
onSuccess: (data, variables) => {
queryClient.invalidateQueries({
queryKey: $api.queryOptions(
"get",
"/v1/transcripts/{transcript_id}",
{
params: {
path: { transcript_id: variables.params.path.transcript_id },
},
},
).queryKey,
});
},
onError: (error) => {
setError(error as Error, "There was an error uploading the audio file");
},
},
);
}
export function useTranscriptWaveform(transcriptId: string | null) {
return $api.useQuery(
"get",
"/v1/transcripts/{transcript_id}/audio/waveform",
{
params: {
path: { transcript_id: transcriptId! },
},
},
{
enabled: !!transcriptId,
},
);
}
export function useTranscriptMP3(transcriptId: string | null) {
const { isAuthenticated } = useAuthReady();
return $api.useQuery(
"get",
"/v1/transcripts/{transcript_id}/audio/mp3",
{
params: {
path: { transcript_id: transcriptId! },
},
},
{
enabled: !!transcriptId && isAuthenticated,
},
);
}
export function useTranscriptTopics(transcriptId: string | null) {
return $api.useQuery(
"get",
"/v1/transcripts/{transcript_id}/topics",
{
params: {
path: { transcript_id: transcriptId || "" },
},
},
{
enabled: !!transcriptId,
},
);
}
export function useTranscriptTopicsWithWords(transcriptId: string | null) {
const { isAuthenticated } = useAuthReady();
return $api.useQuery(
"get",
"/v1/transcripts/{transcript_id}/topics/with-words",
{
params: {
path: { transcript_id: transcriptId || "" },
},
},
{
enabled: !!transcriptId && isAuthenticated,
},
);
}
export function useTranscriptTopicsWithWordsPerSpeaker(
transcriptId: string | null,
topicId: string | null,
) {
const { isAuthenticated } = useAuthReady();
return $api.useQuery(
"get",
"/v1/transcripts/{transcript_id}/topics/{topic_id}/words-per-speaker",
{
params: {
path: {
transcript_id: transcriptId || "",
topic_id: topicId || "",
},
},
},
{
enabled: !!transcriptId && !!topicId && isAuthenticated,
},
);
}
export function useTranscriptParticipants(transcriptId: string | null) {
const { isAuthenticated } = useAuthReady();
return $api.useQuery(
"get",
"/v1/transcripts/{transcript_id}/participants",
{
params: {
path: { transcript_id: transcriptId || "" },
},
},
{
enabled: !!transcriptId && isAuthenticated,
},
);
}
export function useTranscriptParticipantUpdate() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation(
"patch",
"/v1/transcripts/{transcript_id}/participants/{participant_id}",
{
onSuccess: (data, variables) => {
queryClient.invalidateQueries({
queryKey: $api.queryOptions(
"get",
"/v1/transcripts/{transcript_id}/participants",
{
params: {
path: { transcript_id: variables.params.path.transcript_id },
},
},
).queryKey,
});
},
onError: (error) => {
setError(error as Error, "There was an error updating the participant");
},
},
);
}
export function useTranscriptParticipantCreate() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation(
"post",
"/v1/transcripts/{transcript_id}/participants",
{
onSuccess: (data, variables) => {
queryClient.invalidateQueries({
queryKey: $api.queryOptions(
"get",
"/v1/transcripts/{transcript_id}/participants",
{
params: {
path: { transcript_id: variables.params.path.transcript_id },
},
},
).queryKey,
});
},
onError: (error) => {
setError(error as Error, "There was an error creating the participant");
},
},
);
}
export function useTranscriptParticipantDelete() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation(
"delete",
"/v1/transcripts/{transcript_id}/participants/{participant_id}",
{
onSuccess: (data, variables) => {
queryClient.invalidateQueries({
queryKey: $api.queryOptions(
"get",
"/v1/transcripts/{transcript_id}/participants",
{
params: {
path: { transcript_id: variables.params.path.transcript_id },
},
},
).queryKey,
});
},
onError: (error) => {
setError(error as Error, "There was an error deleting the participant");
},
},
);
}
export function useTranscriptSpeakerAssign() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation(
"patch",
"/v1/transcripts/{transcript_id}/speaker/assign",
{
onSuccess: (data, variables) => {
queryClient.invalidateQueries({
queryKey: $api.queryOptions(
"get",
"/v1/transcripts/{transcript_id}",
{
params: {
path: { transcript_id: variables.params.path.transcript_id },
},
},
).queryKey,
});
queryClient.invalidateQueries({
queryKey: $api.queryOptions(
"get",
"/v1/transcripts/{transcript_id}/participants",
{
params: {
path: { transcript_id: variables.params.path.transcript_id },
},
},
).queryKey,
});
},
onError: (error) => {
setError(error as Error, "There was an error assigning the speaker");
},
},
);
}
export function useTranscriptSpeakerMerge() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation(
"patch",
"/v1/transcripts/{transcript_id}/speaker/merge",
{
onSuccess: (data, variables) => {
queryClient.invalidateQueries({
queryKey: $api.queryOptions(
"get",
"/v1/transcripts/{transcript_id}",
{
params: {
path: { transcript_id: variables.params.path.transcript_id },
},
},
).queryKey,
});
queryClient.invalidateQueries({
queryKey: $api.queryOptions(
"get",
"/v1/transcripts/{transcript_id}/participants",
{
params: {
path: { transcript_id: variables.params.path.transcript_id },
},
},
).queryKey,
});
},
onError: (error) => {
setError(error as Error, "There was an error merging speakers");
},
},
);
}
export function useMeetingAudioConsent() {
const { setError } = useError();
return $api.useMutation("post", "/v1/meetings/{meeting_id}/consent", {
onError: (error) => {
setError(error as Error, "There was an error recording consent");
},
});
}
export function useTranscriptWebRTC() {
const { setError } = useError();
return $api.useMutation(
"post",
"/v1/transcripts/{transcript_id}/record/webrtc",
{
onError: (error) => {
setError(error as Error, "There was an error with WebRTC connection");
},
},
);
}
export function useTranscriptCreate() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation("post", "/v1/transcripts", {
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: ["get", "/v1/transcripts/search"],
});
},
onError: (error) => {
setError(error as Error, "There was an error creating the transcript");
},
});
}
export function useRoomsCreateMeeting() {
const { setError } = useError();
const queryClient = useQueryClient();
return $api.useMutation("post", "/v1/rooms/{room_name}/meeting", {
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: $api.queryOptions("get", "/v1/rooms").queryKey,
});
},
onError: (error) => {
setError(error as Error, "There was an error creating the meeting");
},
});
}

12
www/app/lib/array.ts Normal file
View File

@@ -0,0 +1,12 @@
export type NonEmptyArray<T> = [T, ...T[]];
export const isNonEmptyArray = <T>(arr: T[]): arr is NonEmptyArray<T> =>
arr.length > 0;
export const assertNonEmptyArray = <T>(
arr: T[],
err?: string,
): NonEmptyArray<T> => {
if (isNonEmptyArray(arr)) {
return arr;
}
throw new Error(err ?? "Expected non-empty array");
};

167
www/app/lib/auth.ts
View File

@@ -1,157 +1,20 @@
// import { kv } from "@vercel/kv";
import Redlock, { ResourceLockedError } from "redlock";
import { AuthOptions } from "next-auth";
import AuthentikProvider from "next-auth/providers/authentik";
import { JWT } from "next-auth/jwt";
import { JWTWithAccessToken, CustomSession } from "./types";
import Redis from "ioredis";
import { assertExistsAndNonEmptyString } from "./utils";
const PRETIMEOUT = 60; // seconds before token expires to refresh it
const DEFAULT_REDIS_KEY_TIMEOUT = 60 * 60 * 24 * 30; // 30 days (refresh token expires in 30 days)
const kv = new Redis(process.env.KV_URL || "", {
tls: {},
});
const redlock = new Redlock([kv], {});
export const REFRESH_ACCESS_TOKEN_ERROR = "RefreshAccessTokenError" as const;
// 4 min is 1 min less than default authentic value. here we assume that authentic won't be set to access tokens < 4 min
export const REFRESH_ACCESS_TOKEN_BEFORE = 4 * 60 * 1000;
redlock.on("error", (error) => {
if (error instanceof ResourceLockedError) {
return;
}
// Log all other errors.
console.error(error);
});
export const authOptions: AuthOptions = {
providers: [
AuthentikProvider({
clientId: process.env.AUTHENTIK_CLIENT_ID as string,
clientSecret: process.env.AUTHENTIK_CLIENT_SECRET as string,
issuer: process.env.AUTHENTIK_ISSUER,
authorization: {
params: {
scope: "openid email profile offline_access",
},
},
}),
],
session: {
strategy: "jwt",
},
callbacks: {
async jwt({ token, account, user }) {
const extendedToken = token as JWTWithAccessToken;
if (account && user) {
// called only on first login
// XXX account.expires_in used in example is not defined for authentik backend, but expires_at is
const expiresAt = (account.expires_at as number) - PRETIMEOUT;
const jwtToken = {
...extendedToken,
accessToken: account.access_token,
accessTokenExpires: expiresAt * 1000,
refreshToken: account.refresh_token,
};
kv.set(
`token:${jwtToken.sub}`,
JSON.stringify(jwtToken),
"EX",
DEFAULT_REDIS_KEY_TIMEOUT,
);
return jwtToken;
}
if (Date.now() < extendedToken.accessTokenExpires) {
return token;
}
// access token has expired, try to update it
return await redisLockedrefreshAccessToken(token);
},
async session({ session, token }) {
const extendedToken = token as JWTWithAccessToken;
const customSession = session as CustomSession;
customSession.accessToken = extendedToken.accessToken;
customSession.accessTokenExpires = extendedToken.accessTokenExpires;
customSession.error = extendedToken.error;
customSession.user = {
id: extendedToken.sub,
name: extendedToken.name,
email: extendedToken.email,
};
return customSession;
},
},
export const shouldRefreshToken = (accessTokenExpires: number): boolean => {
const timeLeft = accessTokenExpires - Date.now();
return timeLeft < REFRESH_ACCESS_TOKEN_BEFORE;
};
async function redisLockedrefreshAccessToken(token: JWT) {
return await redlock.using(
[token.sub as string, "jwt-refresh"],
5000,
async () => {
const redisToken = await kv.get(`token:${token.sub}`);
const currentToken = JSON.parse(
redisToken as string,
) as JWTWithAccessToken;
export const LOGIN_REQUIRED_PAGES = [
"/transcripts/[!new]",
"/browse(.*)",
"/rooms(.*)",
];
// if there is multiple requests for the same token, it may already have been refreshed
if (Date.now() < currentToken.accessTokenExpires) {
return currentToken;
}
// now really do the request
const newToken = await refreshAccessToken(currentToken);
await kv.set(
`token:${currentToken.sub}`,
JSON.stringify(newToken),
"EX",
DEFAULT_REDIS_KEY_TIMEOUT,
);
return newToken;
},
);
}
async function refreshAccessToken(token: JWT): Promise<JWTWithAccessToken> {
try {
const url = `${process.env.AUTHENTIK_REFRESH_TOKEN_URL}`;
const options = {
headers: {
"Content-Type": "application/x-www-form-urlencoded",
},
body: new URLSearchParams({
client_id: process.env.AUTHENTIK_CLIENT_ID as string,
client_secret: process.env.AUTHENTIK_CLIENT_SECRET as string,
grant_type: "refresh_token",
refresh_token: token.refreshToken as string,
}).toString(),
method: "POST",
};
const response = await fetch(url, options);
if (!response.ok) {
console.error(
new Date().toISOString(),
"Failed to refresh access token. Response status:",
response.status,
);
const responseBody = await response.text();
console.error(new Date().toISOString(), "Response body:", responseBody);
throw new Error(`Failed to refresh access token: ${response.statusText}`);
}
const refreshedTokens = await response.json();
return {
...token,
accessToken: refreshedTokens.access_token,
accessTokenExpires:
Date.now() + (refreshedTokens.expires_in - PRETIMEOUT) * 1000,
refreshToken: refreshedTokens.refresh_token,
};
} catch (error) {
console.error("Error refreshing access token", error);
return {
...token,
error: "RefreshAccessTokenError",
} as JWTWithAccessToken;
}
}
export const PROTECTED_PAGES = new RegExp(
LOGIN_REQUIRED_PAGES.map((page) => `^${page}$`).join("|"),
);

245
www/app/lib/authBackend.ts Normal file
View File

@@ -0,0 +1,245 @@
import { AuthOptions } from "next-auth";
import AuthentikProvider from "next-auth/providers/authentik";
import type { JWT } from "next-auth/jwt";
import { JWTWithAccessToken, CustomSession } from "./types";
import {
assertExists,
assertExistsAndNonEmptyString,
assertNotExists,
} from "./utils";
import {
REFRESH_ACCESS_TOKEN_BEFORE,
REFRESH_ACCESS_TOKEN_ERROR,
shouldRefreshToken,
} from "./auth";
import {
getTokenCache,
setTokenCache,
deleteTokenCache,
} from "./redisTokenCache";
import { tokenCacheRedis, redlock } from "./redisClient";
import { isBuildPhase } from "./next";
import { sequenceThrows } from "./errorUtils";
import { featureEnabled } from "./features";
const TOKEN_CACHE_TTL = REFRESH_ACCESS_TOKEN_BEFORE;
const getAuthentikClientId = () =>
assertExistsAndNonEmptyString(
process.env.AUTHENTIK_CLIENT_ID,
"AUTHENTIK_CLIENT_ID required",
);
const getAuthentikClientSecret = () =>
assertExistsAndNonEmptyString(
process.env.AUTHENTIK_CLIENT_SECRET,
"AUTHENTIK_CLIENT_SECRET required",
);
const getAuthentikRefreshTokenUrl = () =>
assertExistsAndNonEmptyString(
process.env.AUTHENTIK_REFRESH_TOKEN_URL,
"AUTHENTIK_REFRESH_TOKEN_URL required",
);
export const authOptions = (): AuthOptions =>
featureEnabled("requireLogin")
? {
providers: [
AuthentikProvider({
...(() => {
const [clientId, clientSecret] = sequenceThrows(
getAuthentikClientId,
getAuthentikClientSecret,
);
return {
clientId,
clientSecret,
};
})(),
issuer: process.env.AUTHENTIK_ISSUER,
authorization: {
params: {
scope: "openid email profile offline_access",
},
},
}),
],
session: {
strategy: "jwt",
},
callbacks: {
async jwt({ token, account, user }) {
if (account && !account.access_token) {
await deleteTokenCache(tokenCacheRedis, `token:${token.sub}`);
}
if (account && user) {
// called only on first login
// XXX account.expires_in used in example is not defined for authentik backend, but expires_at is
if (account.access_token) {
const expiresAtS = assertExists(account.expires_at);
const expiresAtMs = expiresAtS * 1000;
const jwtToken: JWTWithAccessToken = {
...token,
accessToken: account.access_token,
accessTokenExpires: expiresAtMs,
refreshToken: account.refresh_token,
};
if (jwtToken.error) {
await deleteTokenCache(tokenCacheRedis, `token:${token.sub}`);
} else {
assertNotExists(
jwtToken.error,
`panic! trying to cache token with error in jwt: ${jwtToken.error}`,
);
await setTokenCache(tokenCacheRedis, `token:${token.sub}`, {
token: jwtToken,
timestamp: Date.now(),
});
return jwtToken;
}
}
}
const currentToken = await getTokenCache(
tokenCacheRedis,
`token:${token.sub}`,
);
console.debug(
"currentToken from cache",
JSON.stringify(currentToken, null, 2),
"will be returned?",
currentToken &&
!shouldRefreshToken(currentToken.token.accessTokenExpires),
);
if (
currentToken &&
!shouldRefreshToken(currentToken.token.accessTokenExpires)
) {
return currentToken.token;
}
// access token has expired, try to update it
return await lockedRefreshAccessToken(token);
},
async session({ session, token }) {
const extendedToken = token as JWTWithAccessToken;
return {
...session,
accessToken: extendedToken.accessToken,
accessTokenExpires: extendedToken.accessTokenExpires,
error: extendedToken.error,
user: {
id: assertExists(extendedToken.sub),
name: extendedToken.name,
email: extendedToken.email,
},
} satisfies CustomSession;
},
},
}
: {
providers: [],
};
async function lockedRefreshAccessToken(
token: JWT,
): Promise<JWTWithAccessToken> {
const lockKey = `${token.sub}-lock`;
return redlock
.using([lockKey], 10000, async () => {
const cached = await getTokenCache(tokenCacheRedis, `token:${token.sub}`);
if (cached)
console.debug(
"received cached token. to delete?",
Date.now() - cached.timestamp > TOKEN_CACHE_TTL,
);
else console.debug("no cached token received");
if (cached) {
if (Date.now() - cached.timestamp > TOKEN_CACHE_TTL) {
await deleteTokenCache(tokenCacheRedis, `token:${token.sub}`);
} else if (!shouldRefreshToken(cached.token.accessTokenExpires)) {
console.debug("returning cached token", cached.token);
return cached.token;
}
}
const currentToken = cached?.token || (token as JWTWithAccessToken);
const newToken = await refreshAccessToken(currentToken);
console.debug("current token during refresh", currentToken);
console.debug("new token during refresh", newToken);
if (newToken.error) {
await deleteTokenCache(tokenCacheRedis, `token:${token.sub}`);
return newToken;
}
assertNotExists(
newToken.error,
`panic! trying to cache token with error during refresh: ${newToken.error}`,
);
await setTokenCache(tokenCacheRedis, `token:${token.sub}`, {
token: newToken,
timestamp: Date.now(),
});
return newToken;
})
.catch((e) => {
console.error("error refreshing token", e);
deleteTokenCache(tokenCacheRedis, `token:${token.sub}`).catch((e) => {
console.error("error deleting errored token", e);
});
return {
...token,
error: REFRESH_ACCESS_TOKEN_ERROR,
} as JWTWithAccessToken;
});
}
async function refreshAccessToken(token: JWT): Promise<JWTWithAccessToken> {
const [url, clientId, clientSecret] = sequenceThrows(
getAuthentikRefreshTokenUrl,
getAuthentikClientId,
getAuthentikClientSecret,
);
try {
const options = {
headers: {
"Content-Type": "application/x-www-form-urlencoded",
},
body: new URLSearchParams({
client_id: clientId,
client_secret: clientSecret,
grant_type: "refresh_token",
refresh_token: token.refreshToken as string,
}).toString(),
method: "POST",
};
const response = await fetch(url, options);
if (!response.ok) {
console.error(
new Date().toISOString(),
"Failed to refresh access token. Response status:",
response.status,
);
const responseBody = await response.text();
console.error(new Date().toISOString(), "Response body:", responseBody);
throw new Error(`Failed to refresh access token: ${response.statusText}`);
}
const refreshedTokens = await response.json();
return {
...token,
accessToken: refreshedTokens.access_token,
accessTokenExpires: Date.now() + refreshedTokens.expires_in * 1000,
refreshToken: refreshedTokens.refresh_token,
};
} catch (error) {
console.error("Error refreshing access token", error);
return {
...token,
error: REFRESH_ACCESS_TOKEN_ERROR,
} as JWTWithAccessToken;
}
}

48
www/app/lib/edgeConfig.ts
View File

@@ -1,48 +0,0 @@
import { get } from "@vercel/edge-config";
import { isDevelopment } from "./utils";
type EdgeConfig = {
[domainWithDash: string]: {
features: {
[featureName in
| "requireLogin"
| "privacy"
| "browse"
| "sendToZulip"]: boolean;
};
auth_callback_url: string;
websocket_url: string;
api_url: string;
};
};
export type DomainConfig = EdgeConfig["domainWithDash"];
// Edge config main keys can only be alphanumeric and _ or -
export function edgeKeyToDomain(key: string) {
return key.replaceAll("_", ".");
}
export function edgeDomainToKey(domain: string) {
return domain.replaceAll(".", "_");
}
// get edge config server-side (prefer DomainContext when available), domain is the hostname
export async function getConfig() {
const domain = new URL(process.env.NEXT_PUBLIC_SITE_URL!).hostname;
if (process.env.NEXT_PUBLIC_ENV === "development") {
return require("../../config").localConfig;
}
let config = await get(edgeDomainToKey(domain));
if (typeof config !== "object") {
console.warn("No config for this domain, falling back to default");
config = await get(edgeDomainToKey("default"));
}
if (typeof config !== "object") throw Error("Error fetching config");
return config as DomainConfig;
}

42
www/app/lib/errorUtils.ts
View File

@@ -1,4 +1,6 @@
function shouldShowError(error: Error | null | undefined) {
import { isNonEmptyArray, NonEmptyArray } from "./array";
export function shouldShowError(error: Error | null | undefined) {
if (
error?.name == "ResponseError" &&
(error["response"].status == 404 || error["response"].status == 403)
@@ -8,4 +10,40 @@ function shouldShowError(error: Error | null | undefined) {
return true;
}
export { shouldShowError };
const defaultMergeErrors = (ex: NonEmptyArray<unknown>): unknown => {
try {
return new Error(
ex
.map((e) =>
e ? (e.toString ? e.toString() : JSON.stringify(e)) : `${e}`,
)
.join("\n"),
);
} catch (e) {
console.error("Error merging errors:", e);
return ex[0];
}
};
type ReturnTypes<T extends readonly (() => any)[]> = {
[K in keyof T]: T[K] extends () => infer R ? R : never;
};
// sequence semantic for "throws"
// calls functions passed and collects its thrown values
export function sequenceThrows<Fns extends readonly (() => any)[]>(
...fs: Fns
): ReturnTypes<Fns> {
const results: unknown[] = [];
const errors: unknown[] = [];
for (const f of fs) {
try {
results.push(f());
} catch (e) {
errors.push(e);
}
}
if (errors.length) throw defaultMergeErrors(errors as NonEmptyArray<unknown>);
return results as ReturnTypes<Fns>;
}

Some files were not shown because too many files have changed in this diff Show More