mirror of
https://github.com/Monadical-SAS/reflector.git
synced 2025-12-20 20:29:06 +00:00
9fe261406c
* initial * add LLM features * update LLM logic * update llm functions: change control flow * add generation config * update return types * update processors and tests * update rtc_offer * revert new title processor change * fix unit tests * add comments and fix HTTP 500 * adjust prompt * test with reflector app * revert new event for final title * update * move onus onto processors * move onus onto processors * stash * add provision for gen config * dynamically pack the LLM input using context length * tune final summary params * update consolidated class structures * update consolidated class structures * update precommit * add broadcast processors * working baseline * Organize LLMParams * minor fixes * minor fixes * minor fixes * fix unit tests * fix unit tests * fix unit tests * update tests * update tests * edit pipeline response events * update summary return types * configure tests * alembic db migration * change LLM response flow * edit main llm functions * edit main llm functions * change llm name and gen cf * Update transcript_topic_detector.py * PR review comments * checkpoint before db event migration * update DB migration of past events * update DB migration of past events * edit LLM classes * Delete unwanted file * remove List typing * remove List typing * update oobabooga API call * topic enhancements * update UI event handling * move ensure_casing to llm base * update tests * update tests
Reflector GPU implementation - Transcription and LLM
This repository hold an API for the GPU implementation of the Reflector API service, and use Modal.com
reflector_llm.py- LLM APIreflector_transcriber.py- Transcription API
Modal.com deployment
Create a modal secret, and name it reflector-gpu.
It should contain an REFLECTOR_APIKEY environment variable with a value.
The deployment is done using Modal.com service.
$ modal deploy reflector_transcriber.py
...
└── 🔨 Created web => https://xxxx--reflector-transcriber-web.modal.run
$ modal deploy reflector_llm.py
...
└── 🔨 Created web => https://xxxx--reflector-llm-web.modal.run
Then in your reflector api configuration .env, you can set theses keys:
TRANSCRIPT_BACKEND=modal
TRANSCRIPT_URL=https://xxxx--reflector-transcriber-web.modal.run
TRANSCRIPT_MODAL_API_KEY=REFLECTOR_APIKEY
LLM_BACKEND=modal
LLM_URL=https://xxxx--reflector-llm-web.modal.run
LLM_MODAL_API_KEY=REFLECTOR_APIKEY
API
Authentication must be passed with the Authorization header, using the bearer scheme.
Authorization: bearer <REFLECTOR_APIKEY>
Warmup (both)
POST /warmup
response
{
"status": "ok"
}
LLM
POST /llm
request
{
"prompt": "xxx"
}
response
{
"text": "xxx completed"
}
Transcription
POST /transcribe
request (multipart/form-data)
file- audio filelanguage- language code (e.g.en)
response
{
"text": "xxx",
"words": [
{"text": "xxx", "start": 0.0, "end": 1.0}
]
}