reflector/docs/jitsi.md

Jitsi Integration for Reflector

This document contains research and planning notes for integrating Jitsi Meet as a replacement for Whereby in Reflector.

Overview

Jitsi Meet is an open-source video conferencing solution that can replace Whereby in Reflector, providing:

• Cost reduction (no per-minute charges)
• Direct recording access via Jibri
• Real-time event webhooks
• Full customization and control

Current Whereby Integration Analysis

Architecture

1. Room Creation: User creates a "room" template in Reflector DB with settings
2. Meeting Creation: /rooms/{room_name}/meeting endpoint calls Whereby API to create meeting
3. Recording: Whereby handles recording automatically to S3 bucket
4. Webhooks: Whereby sends events for participant tracking

Database Structure

# Room = Template/Configuration
class Room:
    id, name, user_id
    recording_type, recording_trigger  # cloud, automatic-2nd-participant
    webhook_url, webhook_secret

# Meeting = Actual Whereby Meeting Instance
class Meeting:
    id  # Whereby meetingId
    room_name  # Generated by Whereby
    room_url, host_room_url  # Whereby URLs
    num_clients  # Updated via webhooks

Jitsi Components

Core Architecture

• Jitsi Meet: Web frontend (Next.js + React)
• Prosody: XMPP server for messaging/rooms
• Jicofo: Conference focus (orchestration)
• JVB: Videobridge (media routing)
• Jibri: Recording service
• Jigasi: SIP gateway (optional, for phone dial-in)

Exposure Requirements

• Web service: 443/80 (frontend)
• JVB: 10000/UDP (media streams) - MUST EXPOSE
• Prosody: 5280 (BOSH/WebSocket) - can proxy via web
• Jicofo, Jibri, Jigasi: Internal only

Recording with Jibri

How Jibri Works

• Each Jibri instance handles one recording at a time
• Records mixed audio/video to MP4 format
• Uses Chrome headless + ffmpeg for capture
• Supports finalize scripts for post-processing

Jibri Pool for Scaling

• Multiple Jibri instances join "jibribrewery" MUC
• Jicofo distributes recording requests to available instances
• Automatic load balancing and failover

# Multiple Jibri instances
jibri1:
  environment:
    - JIBRI_INSTANCE_ID=jibri1
    - JIBRI_BREWERY_MUC=jibribrewery

jibri2:
  environment:
    - JIBRI_INSTANCE_ID=jibri2
    - JIBRI_BREWERY_MUC=jibribrewery

Recording Automation Options

1. Environment Variables: ENABLE_RECORDING=1, AUTO_RECORDING=1
2. URL Parameters: ?config.autoRecord=true
3. JWT Token: Include recording permissions in JWT
4. API Control: api.executeCommand('startRecording')

Post-Processing Integration

#!/bin/bash
# finalize.sh - runs after recording completion
RECORDING_FILE=$1
MEETING_METADATA=$2
ROOM_NAME=$3

# Copy to Reflector-accessible location
cp "$RECORDING_FILE" /shared/reflector-uploads/

# Trigger Reflector processing
curl -X POST "http://reflector-api:8000/v1/transcripts/process" \
  -H "Content-Type: application/json" \
  -d "{
    \"file_path\": \"/shared/reflector-uploads/$(basename $RECORDING_FILE)\",
    \"room_name\": \"$ROOM_NAME\",
    \"source\": \"jitsi\"
  }"

React Integration

Official React SDK

npm i @jitsi/react-sdk

import { JitsiMeeting } from '@jitsi/react-sdk'

<JitsiMeeting
  room="meeting-room"
  serverURL="https://your-jitsi.domain"
  jwt="your-jwt-token"
  config={{
    startWithAudioMuted: true,
    fileRecordingsEnabled: true,
    autoRecord: true
  }}
  onParticipantJoined={(participant) => {
    // Track participant events
  }}
  onRecordingStatusChanged={(status) => {
    // Handle recording events
  }}
/>

Authentication & Room Control

JWT-Based Access Control

def generate_jitsi_jwt(payload):
    return jwt.encode({
        "aud": "jitsi",
        "iss": "reflector",
        "sub": "reflector-user",
        "room": payload["room"],
        "exp": int(payload["exp"].timestamp()),
        "context": {
            "user": {
                "name": payload["user_name"],
                "moderator": payload.get("moderator", False)
            },
            "features": {
                "recording": payload.get("recording", True)
            }
        }
    }, JITSI_JWT_SECRET)

Prevent Anonymous Room Creation

# Environment configuration
ENABLE_AUTH=1
ENABLE_GUESTS=0
AUTH_TYPE=jwt
JWT_APP_ID=reflector
JWT_APP_SECRET=your-secret-key

Webhook Integration

Real-time Events via Prosody

Custom event-sync module can send webhooks for:

• Participant join/leave
• Recording start/stop
• Room creation/destruction
• Mute/unmute events

-- mod_event_sync.lua
module:hook("muc-occupant-joined", function(event)
    send_event({
        type = "participant_joined",
        room = event.room.jid,
        participant = {
            nick = event.occupant.nick,
            jid = event.occupant.jid,
        },
        timestamp = os.time(),
    });
end);

Jibri Recording Webhooks

# Environment variable
JIBRI_WEBHOOK_SUBSCRIBERS=https://your-reflector.com/webhooks/jibri

Proposed Reflector Integration

Modified Database Schema

class Meeting(BaseModel):
    id: str  # Our generated meeting ID
    room_name: str  # Generated: reflector-{room.name}-{timestamp}
    room_url: str  # https://jitsi.domain/room_name?jwt=token
    host_room_url: str  # Same but with moderator JWT
    # Add Jitsi-specific fields
    jitsi_jwt: str  # JWT token
    jitsi_room_id: str  # Internal room identifier
    recording_status: str  # pending, recording, completed
    recording_file_path: Optional[str]

API Replacement

# Replace whereby.py with jitsi.py
async def create_meeting(room_name_prefix: str, end_date: datetime, room: Room):
    # Generate unique room name
    jitsi_room = f"reflector-{room.name}-{int(time.time())}"

    # Generate JWT tokens
    user_jwt = generate_jwt(room=jitsi_room, moderator=False, exp=end_date)
    host_jwt = generate_jwt(room=jitsi_room, moderator=True, exp=end_date)

    return {
        "meetingId": generate_uuid4(),  # Our ID
        "roomName": jitsi_room,
        "roomUrl": f"https://jitsi.domain/{jitsi_room}?jwt={user_jwt}",
        "hostRoomUrl": f"https://jitsi.domain/{jitsi_room}?jwt={host_jwt}",
        "startDate": datetime.now().isoformat(),
        "endDate": end_date.isoformat(),
    }

Webhook Endpoints

# Replace whereby webhook with jitsi webhooks
@router.post("/jitsi/events")
async def jitsi_events_webhook(event_data: dict):
    event_type = event_data.get("event")
    room_name = event_data.get("room", "").split("@")[0]

    meeting = await Meeting.get_by_room(room_name)

    if event_type == "muc-occupant-joined":
        # Update participant count
        meeting.num_clients += 1

    elif event_type == "jibri-recording-on":
        meeting.recording_status = "recording"

    elif event_type == "jibri-recording-off":
        meeting.recording_status = "processing"
        await process_meeting_recording.delay(meeting.id)

@router.post("/jibri/recording-complete")
async def recording_complete(data: dict):
    # Handle finalize script webhook
    room_name = data.get("room_name")
    file_path = data.get("file_path")

    meeting = await Meeting.get_by_room(room_name)
    meeting.recording_file_path = file_path
    meeting.recording_status = "completed"

    # Start Reflector processing
    await process_recording_for_transcription(meeting.id, file_path)

Deployment with Docker

Official docker-jitsi-meet

# Download official release
wget $(wget -q -O - https://api.github.com/repos/jitsi/docker-jitsi-meet/releases/latest | grep zip | cut -d\" -f4)

# Setup
mkdir -p ~/.jitsi-meet-cfg/{web,transcripts,prosody/config,prosody/prosody-plugins-custom,jicofo,jvb,jigasi,jibri}
./gen-passwords.sh  # Generate secure passwords
docker compose up -d

Coolify Integration

services:
  web:
    ports: ["80:80", "443:443"]
  jvb:
    ports: ["10000:10000/udp"]  # Must expose for media
  jibri1:
    environment:
      - JIBRI_INSTANCE_ID=jibri1
      - JIBRI_FINALIZE_RECORDING_SCRIPT_PATH=/config/finalize.sh
  jibri2:
    environment:
      - JIBRI_INSTANCE_ID=jibri2

Benefits vs Whereby

Cost & Control

✅ No per-minute charges - significant cost savings ✅ Full recording control - direct file access ✅ Custom branding - complete UI control ✅ Self-hosted - no vendor lock-in

Technical Advantages

✅ Real-time events - immediate webhook notifications ✅ Rich participant metadata - detailed tracking ✅ JWT security - token-based access with expiration ✅ Multiple recording formats - audio-only options ✅ Scalable architecture - horizontal Jibri scaling

Integration Benefits

✅ Same API surface - minimal changes to existing code ✅ React SDK - better frontend integration ✅ Direct processing - no S3 download delays ✅ Event-driven architecture - better real-time capabilities

Implementation Plan

1. Deploy Jitsi Stack - Set up docker-jitsi-meet with multiple Jibri instances
2. Create jitsi.py - Replace whereby.py with Jitsi API functions
3. Update Database - Add Jitsi-specific fields to Meeting model
4. Webhook Integration - Replace Whereby webhooks with Jitsi events
5. Frontend Updates - Replace Whereby embed with Jitsi React SDK
6. Testing & Migration - Gradual rollout with fallback to Whereby

Recording Limitations & Considerations

Current Limitations

• Mixed audio only - Jibri doesn't separate participant tracks natively
• One recording per Jibri - requires multiple instances for concurrent recordings
• Chrome dependency - Jibri uses headless Chrome for recording

Metadata Capabilities

✅ Participant join/leave timestamps - via webhooks ✅ Speaking time tracking - via audio level events ✅ Meeting duration - precise timing ✅ Room-specific data - custom metadata in JWT

Alternative Recording Methods

• Local recording - browser-based, per-participant
• Custom recording - lib-jitsi-meet for individual streams
• Third-party solutions - Recall.ai, Otter.ai integrations

Security Considerations

JWT Configuration

• Room-specific tokens - limit access to specific rooms
• Time-based expiration - automatic cleanup
• Feature permissions - control recording, moderation rights
• User identification - embed user metadata in tokens

Access Control

• No anonymous rooms - all rooms require valid JWT
• API-only creation - prevent direct room access
• Webhook verification - HMAC signature validation

Next Steps

1. Deploy test Jitsi instance - validate recording pipeline
2. Prototype jitsi.py - create equivalent API functions
3. Test webhook integration - ensure event delivery works
4. Performance testing - validate multiple concurrent recordings
5. Migration strategy - plan gradual transition from Whereby

---

This document serves as the comprehensive planning and research notes for Jitsi integration in Reflector. It should be updated as implementation progresses and new insights are discovered.