reflector/docs/jitsi.md

# Jitsi Integration for Reflector

This document contains research and planning notes for integrating Jitsi Meet as a replacement for Whereby in Reflector.

## Overview

Jitsi Meet is an open-source video conferencing solution that can replace Whereby in Reflector, providing:
- Cost reduction (no per-minute charges)
- Direct recording access via Jibri
- Real-time event webhooks
- Full customization and control

## Current Whereby Integration Analysis

### Architecture
1. **Room Creation**: User creates a "room" template in Reflector DB with settings
2. **Meeting Creation**: `/rooms/{room_name}/meeting` endpoint calls Whereby API to create meeting
3. **Recording**: Whereby handles recording automatically to S3 bucket
4. **Webhooks**: Whereby sends events for participant tracking

### Database Structure
```python
# Room = Template/Configuration
class Room:
    id, name, user_id
    recording_type, recording_trigger  # cloud, automatic-2nd-participant
    webhook_url, webhook_secret

# Meeting = Actual Whereby Meeting Instance
class Meeting:
    id  # Whereby meetingId
    room_name  # Generated by Whereby
    room_url, host_room_url  # Whereby URLs
    num_clients  # Updated via webhooks
```

## Jitsi Components

### Core Architecture
- **Jitsi Meet**: Web frontend (Next.js + React)
- **Prosody**: XMPP server for messaging/rooms
- **Jicofo**: Conference focus (orchestration)
- **JVB**: Videobridge (media routing)
- **Jibri**: Recording service
- **Jigasi**: SIP gateway (optional, for phone dial-in)

### Exposure Requirements
- **Web service**: 443/80 (frontend)
- **JVB**: 10000/UDP (media streams) - **MUST EXPOSE**
- **Prosody**: 5280 (BOSH/WebSocket) - can proxy via web
- **Jicofo, Jibri, Jigasi**: Internal only

## Recording with Jibri

### How Jibri Works
- Each Jibri instance handles **one recording at a time**
- Records mixed audio/video to MP4 format
- Uses Chrome headless + ffmpeg for capture
- Supports finalize scripts for post-processing

### Jibri Pool for Scaling
- Multiple Jibri instances join "jibribrewery" MUC
- Jicofo distributes recording requests to available instances
- Automatic load balancing and failover

```yaml
# Multiple Jibri instances
jibri1:
  environment:
    - JIBRI_INSTANCE_ID=jibri1
    - JIBRI_BREWERY_MUC=jibribrewery

jibri2:
  environment:
    - JIBRI_INSTANCE_ID=jibri2
    - JIBRI_BREWERY_MUC=jibribrewery
```

### Recording Automation Options
1. **Environment Variables**: `ENABLE_RECORDING=1`, `AUTO_RECORDING=1`
2. **URL Parameters**: `?config.autoRecord=true`
3. **JWT Token**: Include recording permissions in JWT
4. **API Control**: `api.executeCommand('startRecording')`

### Post-Processing Integration
```bash
#!/bin/bash
# finalize.sh - runs after recording completion
RECORDING_FILE=$1
MEETING_METADATA=$2
ROOM_NAME=$3

# Copy to Reflector-accessible location
cp "$RECORDING_FILE" /shared/reflector-uploads/

# Trigger Reflector processing
curl -X POST "http://reflector-api:8000/v1/transcripts/process" \
  -H "Content-Type: application/json" \
  -d "{
    \"file_path\": \"/shared/reflector-uploads/$(basename $RECORDING_FILE)\",
    \"room_name\": \"$ROOM_NAME\",
    \"source\": \"jitsi\"
  }"
```

## React Integration

### Official React SDK
```bash
npm i @jitsi/react-sdk
```

```jsx
import { JitsiMeeting } from '@jitsi/react-sdk'

<JitsiMeeting
  room="meeting-room"
  serverURL="https://your-jitsi.domain"
  jwt="your-jwt-token"
  config={{
    startWithAudioMuted: true,
    fileRecordingsEnabled: true,
    autoRecord: true
  }}
  onParticipantJoined={(participant) => {
    // Track participant events
  }}
  onRecordingStatusChanged={(status) => {
    // Handle recording events
  }}
/>
```

## Authentication & Room Control

### JWT-Based Access Control
```python
def generate_jitsi_jwt(payload):
    return jwt.encode({
        "aud": "jitsi",
        "iss": "reflector",
        "sub": "reflector-user",
        "room": payload["room"],
        "exp": int(payload["exp"].timestamp()),
        "context": {
            "user": {
                "name": payload["user_name"],
                "moderator": payload.get("moderator", False)
            },
            "features": {
                "recording": payload.get("recording", True)
            }
        }
    }, JITSI_JWT_SECRET)
```

### Prevent Anonymous Room Creation
```bash
# Environment configuration
ENABLE_AUTH=1
ENABLE_GUESTS=0
AUTH_TYPE=jwt
JWT_APP_ID=reflector
JWT_APP_SECRET=your-secret-key
```

## Webhook Integration

### Real-time Events via Prosody
Custom event-sync module can send webhooks for:
- Participant join/leave
- Recording start/stop
- Room creation/destruction
- Mute/unmute events

```lua
-- mod_event_sync.lua
module:hook("muc-occupant-joined", function(event)
    send_event({
        type = "participant_joined",
        room = event.room.jid,
        participant = {
            nick = event.occupant.nick,
            jid = event.occupant.jid,
        },
        timestamp = os.time(),
    });
end);
```

### Jibri Recording Webhooks
```bash
# Environment variable
JIBRI_WEBHOOK_SUBSCRIBERS=https://your-reflector.com/webhooks/jibri
```

## Proposed Reflector Integration

### Modified Database Schema
```python
class Meeting(BaseModel):
    id: str  # Our generated meeting ID
    room_name: str  # Generated: reflector-{room.name}-{timestamp}
    room_url: str  # https://jitsi.domain/room_name?jwt=token
    host_room_url: str  # Same but with moderator JWT
    # Add Jitsi-specific fields
    jitsi_jwt: str  # JWT token
    jitsi_room_id: str  # Internal room identifier
    recording_status: str  # pending, recording, completed
    recording_file_path: Optional[str]
```

### API Replacement
```python
# Replace whereby.py with jitsi.py
async def create_meeting(room_name_prefix: str, end_date: datetime, room: Room):
    # Generate unique room name
    jitsi_room = f"reflector-{room.name}-{int(time.time())}"

    # Generate JWT tokens
    user_jwt = generate_jwt(room=jitsi_room, moderator=False, exp=end_date)
    host_jwt = generate_jwt(room=jitsi_room, moderator=True, exp=end_date)

    return {
        "meetingId": generate_uuid4(),  # Our ID
        "roomName": jitsi_room,
        "roomUrl": f"https://jitsi.domain/{jitsi_room}?jwt={user_jwt}",
        "hostRoomUrl": f"https://jitsi.domain/{jitsi_room}?jwt={host_jwt}",
        "startDate": datetime.now().isoformat(),
        "endDate": end_date.isoformat(),
    }
```

### Webhook Endpoints
```python
# Replace whereby webhook with jitsi webhooks
@router.post("/jitsi/events")
async def jitsi_events_webhook(event_data: dict):
    event_type = event_data.get("event")
    room_name = event_data.get("room", "").split("@")[0]

    meeting = await Meeting.get_by_room(room_name)

    if event_type == "muc-occupant-joined":
        # Update participant count
        meeting.num_clients += 1

    elif event_type == "jibri-recording-on":
        meeting.recording_status = "recording"

    elif event_type == "jibri-recording-off":
        meeting.recording_status = "processing"
        await process_meeting_recording.delay(meeting.id)

@router.post("/jibri/recording-complete")
async def recording_complete(data: dict):
    # Handle finalize script webhook
    room_name = data.get("room_name")
    file_path = data.get("file_path")

    meeting = await Meeting.get_by_room(room_name)
    meeting.recording_file_path = file_path
    meeting.recording_status = "completed"

    # Start Reflector processing
    await process_recording_for_transcription(meeting.id, file_path)
```

## Deployment with Docker

### Official docker-jitsi-meet
```bash
# Download official release
wget $(wget -q -O - https://api.github.com/repos/jitsi/docker-jitsi-meet/releases/latest | grep zip | cut -d\" -f4)

# Setup
mkdir -p ~/.jitsi-meet-cfg/{web,transcripts,prosody/config,prosody/prosody-plugins-custom,jicofo,jvb,jigasi,jibri}
./gen-passwords.sh  # Generate secure passwords
docker compose up -d
```

### Coolify Integration
```yaml
services:
  web:
    ports: ["80:80", "443:443"]
  jvb:
    ports: ["10000:10000/udp"]  # Must expose for media
  jibri1:
    environment:
      - JIBRI_INSTANCE_ID=jibri1
      - JIBRI_FINALIZE_RECORDING_SCRIPT_PATH=/config/finalize.sh
  jibri2:
    environment:
      - JIBRI_INSTANCE_ID=jibri2
```

## Benefits vs Whereby

### Cost & Control
✅ **No per-minute charges** - significant cost savings
✅ **Full recording control** - direct file access
✅ **Custom branding** - complete UI control
✅ **Self-hosted** - no vendor lock-in

### Technical Advantages
✅ **Real-time events** - immediate webhook notifications
✅ **Rich participant metadata** - detailed tracking
✅ **JWT security** - token-based access with expiration
✅ **Multiple recording formats** - audio-only options
✅ **Scalable architecture** - horizontal Jibri scaling

### Integration Benefits
✅ **Same API surface** - minimal changes to existing code
✅ **React SDK** - better frontend integration
✅ **Direct processing** - no S3 download delays
✅ **Event-driven architecture** - better real-time capabilities

## Implementation Plan

1. **Deploy Jitsi Stack** - Set up docker-jitsi-meet with multiple Jibri instances
2. **Create jitsi.py** - Replace whereby.py with Jitsi API functions
3. **Update Database** - Add Jitsi-specific fields to Meeting model
4. **Webhook Integration** - Replace Whereby webhooks with Jitsi events
5. **Frontend Updates** - Replace Whereby embed with Jitsi React SDK
6. **Testing & Migration** - Gradual rollout with fallback to Whereby

## Recording Limitations & Considerations

### Current Limitations
- **Mixed audio only** - Jibri doesn't separate participant tracks natively
- **One recording per Jibri** - requires multiple instances for concurrent recordings
- **Chrome dependency** - Jibri uses headless Chrome for recording

### Metadata Capabilities
✅ **Participant join/leave timestamps** - via webhooks
✅ **Speaking time tracking** - via audio level events
✅ **Meeting duration** - precise timing
✅ **Room-specific data** - custom metadata in JWT

### Alternative Recording Methods
- **Local recording** - browser-based, per-participant
- **Custom recording** - lib-jitsi-meet for individual streams
- **Third-party solutions** - Recall.ai, Otter.ai integrations

## Security Considerations

### JWT Configuration
- **Room-specific tokens** - limit access to specific rooms
- **Time-based expiration** - automatic cleanup
- **Feature permissions** - control recording, moderation rights
- **User identification** - embed user metadata in tokens

### Access Control
- **No anonymous rooms** - all rooms require valid JWT
- **API-only creation** - prevent direct room access
- **Webhook verification** - HMAC signature validation

## Next Steps

1. **Deploy test Jitsi instance** - validate recording pipeline
2. **Prototype jitsi.py** - create equivalent API functions
3. **Test webhook integration** - ensure event delivery works
4. **Performance testing** - validate multiple concurrent recordings
5. **Migration strategy** - plan gradual transition from Whereby

---

*This document serves as the comprehensive planning and research notes for Jitsi integration in Reflector. It should be updated as implementation progresses and new insights are discovered.*