406164033d
feat: new summary using phi-4 and llama-index ( #519 )
...
* feat: add litellm backend implementation
* refactor: improve generate/completion methods for base LLM
* refactor: remove tokenizer logic
* style: apply code formatting
* fix: remove hallucinations from LLM responses
* refactor: comprehensive LLM and summarization rework
* chore: remove debug code
* feat: add structured output support to LiteLLM
* refactor: apply self-review improvements
* docs: add model structured output comments
* docs: update model structured output comments
* style: apply linting and formatting fixes
* fix: resolve type logic bug
* refactor: apply PR review feedback
* refactor: apply additional PR review feedback
* refactor: apply final PR review feedback
* fix: improve schema passing for LLMs without structured output
* feat: add PR comments and logger improvements
* docs: update README and add HTTP logging
* feat: improve HTTP logging
* feat: add summary chunking functionality
* fix: resolve title generation runtime issues
* refactor: apply self-review improvements
* style: apply linting and formatting
* feat: implement LiteLLM class structure
* style: apply linting and formatting fixes
* docs: env template model name fix
* chore: remove older litellm class
* chore: format
* refactor: simplify OpenAILLM
* refactor: OpenAILLM tokenizer
* refactor: self-review
* refactor: self-review
* refactor: self-review
* chore: format
* chore: remove LLM_USE_STRUCTURED_OUTPUT from envs
* chore: roll back migration lint changes
* chore: roll back migration lint changes
* fix: make summary llm configuration optional for the tests
* fix: missing f-string
* fix: tweak the prompt for summary title
* feat: try llamaindex for summarization
* fix: complete refactor of summary builder using llamaindex and structured output when possible
* fix: separate prompt as constant
* fix: typings
* fix: enhance prompt to prevent mentioning others subject while summarize one
* fix: various changes after self-review
* fix: from igor review
---------
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com >
2025-07-31 15:29:29 -06:00
2a2af5fff2
fix: remove fief out of the source code ( #502 )
...
* fix: remove fief out of the source code
* fix: remove corresponding test about migration
2025-07-21 21:09:05 -06:00
4d21fd1754
refactor: migration from sqlite to postgres with migration script ( #483 )
2025-07-16 19:38:33 -06:00
86ce68651f
build: move to uv ( #488 )
...
* build: move to uv
* build: add packages declaration
* build: move to python 3.12, as sentencespiece does not work on 3.13
* ci: remove pre-commit check, will be done in another branch.
* ci: fix name checkout
* ci: update lock and dockerfile
* test: remove event_loop, not needed in python 3.12
* test: updated test due to av returning AudioFrame with 4096 samples instead of 1024
* build: prevent using fastapi cli, because there is no way to set default port
I don't want to pass --port 1250 every time, so back on previous
approach. I deactivated auto-reload for production.
* ci: remove main.py
* test: fix quirck with httpx
2025-07-16 18:10:11 -06:00
99ff06ff17
OpenAI compatible transcription api
2025-01-20 12:27:58 +01:00
5267ab2d37
feat: retake summary using NousResearch/Hermes-3-Llama-3.1-8B model ( #415 )
...
This feature a new modal endpoint, and a complete new way to build the
summary.
## SummaryBuilder
The summary builder is based on conversational model, where an exchange
between the model and the user is made. This allow more context
inclusion and a better respect of the rules.
It requires an endpoint with OpenAI-like completions endpoint
(/v1/chat/completions)
## vLLM Hermes3
Unlike previous deployment, this one use vLLM, which gives OpenAI-like
completions endpoint out of the box. It could also handle guided JSON
generation, so jsonformer is not needed. But, the model is quite good to
follow JSON schema if asked in the prompt.
## Conversion of long/short into summary builder
The builder is identifying participants, find key subjects, get a
summary for each, then get a quick recap.
The quick recap is used as a short_summary, while the markdown including
the quick recap + key subjects + summaries are used for the
long_summary.
This is why the nextjs component has to be updated, to correctly style
h1 and keep the new line of the markdown.
2024-09-14 02:28:38 +02:00
projects-g
1522d60cbc
Big file upload ( #349 )
2024-01-30 20:51:58 +05:30
projects-g
3d87a13e0f
Upgrade libs to bypass security vulnerabilities ( #344 )
...
update libraries
2023-12-28 11:56:21 +05:30
dependabot[bot]
802a7c5b22
build(deps): bump aiohttp from 3.8.6 to 3.9.0 in /server
...
Bumps [aiohttp](https://github.com/aio-libs/aiohttp ) from 3.8.6 to 3.9.0.
- [Release notes](https://github.com/aio-libs/aiohttp/releases )
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst )
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.8.6...v3.9.0 )
---
updated-dependencies:
- dependency-name: aiohttp
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
2023-12-15 14:46:44 +00:00
e5e1b70213
server: include endpoint to upload a audio/video file
2023-12-12 20:39:15 +01:00
afa8010d29
server: add a way to do profiling on api request by adding profile=1
2023-11-10 14:41:28 +01:00
d0057ae2c4
server: add missing python-jose
2023-11-02 17:39:21 +01:00
07c4d080c2
server: refactor with diarization, logic works
2023-11-02 17:39:21 +01:00
00c06b7971
server: use redis pubsub for interprocess websocket communication
2023-11-02 17:39:21 +01:00
8bebb2a769
server: start moving to an external celery task
2023-11-02 17:39:21 +01:00
c5297be924
gh: use poetry cache from setup-python and remove old deps ( #281 )
...
* gh: use poetry cache from setup-python and remove old deps
* gh: use pipx and not setup-poetry, as per setup-python example
* server: remove pyaudio unused in current reflector
2023-10-13 15:29:54 +02:00
Gokul Mohanarangan
ab41ce90e8
add profanity filter, post-process topic/title
2023-09-21 11:12:00 +05:30
Gokul Mohanarangan
9c9bbced47
add protobuf
2023-09-13 12:57:20 +05:30
Gokul Mohanarangan
7f300b9876
add sentencepiece
2023-09-13 12:55:29 +05:30
projects-g
9fe261406c
Feature additions ( #210 )
...
* initial
* add LLM features
* update LLM logic
* update llm functions: change control flow
* add generation config
* update return types
* update processors and tests
* update rtc_offer
* revert new title processor change
* fix unit tests
* add comments and fix HTTP 500
* adjust prompt
* test with reflector app
* revert new event for final title
* update
* move onus onto processors
* move onus onto processors
* stash
* add provision for gen config
* dynamically pack the LLM input using context length
* tune final summary params
* update consolidated class structures
* update consolidated class structures
* update precommit
* add broadcast processors
* working baseline
* Organize LLMParams
* minor fixes
* minor fixes
* minor fixes
* fix unit tests
* fix unit tests
* fix unit tests
* update tests
* update tests
* edit pipeline response events
* update summary return types
* configure tests
* alembic db migration
* change LLM response flow
* edit main llm functions
* edit main llm functions
* change llm name and gen cf
* Update transcript_topic_detector.py
* PR review comments
* checkpoint before db event migration
* update DB migration of past events
* update DB migration of past events
* edit LLM classes
* Delete unwanted file
* remove List typing
* remove List typing
* update oobabooga API call
* topic enhancements
* update UI event handling
* move ensure_casing to llm base
* update tests
* update tests
2023-09-13 11:26:08 +05:30
60edca6366
server: add prometheus instrumentation
2023-09-12 13:11:13 +02:00
cce8a9137a
server: add basic sql migration
2023-08-29 10:58:27 +02:00
e12f9afe7b
server: implement user authentication (none by default)
2023-08-16 17:24:05 +02:00
857505124f
server: implement data persistence with database
...
Using databases + sqlite/postgresql depending of what you want.
Use DATABASE_URL to configure
Closes #70
2023-08-15 17:40:26 +02:00
Gokul Mohanarangan
af954e2818
resolve review comments
2023-08-10 14:33:46 +05:30
Gokul Mohanarangan
0a11a7f669
change dependency group
2023-08-10 12:16:39 +05:30
Gokul Mohanarangan
7ee049bad6
update deps
2023-08-10 12:09:52 +05:30
Mathieu Virbel
93564bfd89
server: fix stamina missing for old server
2023-08-08 18:31:39 +02:00
Mathieu Virbel
0272904363
Merge
2023-08-08 15:33:55 +02:00
7d40305737
Implement retry that automatically detect httpx and backoff ( #119 )
...
* server: implement retry that automatically detect httpx and backoff
Closes #118
* server: fix formatting
2023-08-08 14:03:36 +02:00
Mathieu Virbel
96f52c631a
api: implement first server API + tests
2023-08-04 20:06:43 +02:00
Mathieu Virbel
20767fde3f
server: start implementing new api
2023-08-04 18:17:02 +02:00
d94e2911c3
Serverless GPU support on banana.dev ( #106 )
...
* serverless: implement banana backend for both audio and LLM
Related to monadical-sas/reflector-gpu-banana project
* serverless: got llm working on banana !
* tests: fixes
* serverless: fix dockerfile to use fastapi server + httpx
2023-08-04 10:24:11 +02:00
4d4292e1b5
server: add sentry to server ( #93 )
...
Closes #91
2023-08-02 11:28:18 +02:00
224afc6f28
fastapi: implement server with same back compatibility as before
2023-07-29 15:59:25 +02:00
Mathieu Virbel
c7f2453090
server: formatting
2023-07-27 18:09:01 +02:00
Mathieu Virbel
fe85005e8e
server: add basic rtc test with local audio and fake llm
2023-07-27 18:04:26 +02:00
Mathieu Virbel
69ba871481
server: refactor to reflector module
...
- replaced loguru to structlog, to get ability of having open tracing later
- moved configuration to pydantic-settings
- merged both secrets.ini and config.ini to .env (check reflector/settings.py)
2023-07-27 15:31:58 +02:00
Mathieu Virbel
094ed696c4
server: reformat whole project using black
2023-07-27 14:08:41 +02:00
Mathieu Virbel
b5e0baa6c8
server: dockerize the server and update documentation
2023-07-27 12:18:49 +02:00
