406164033d
feat: new summary using phi-4 and llama-index ( #519 )
...
* feat: add litellm backend implementation
* refactor: improve generate/completion methods for base LLM
* refactor: remove tokenizer logic
* style: apply code formatting
* fix: remove hallucinations from LLM responses
* refactor: comprehensive LLM and summarization rework
* chore: remove debug code
* feat: add structured output support to LiteLLM
* refactor: apply self-review improvements
* docs: add model structured output comments
* docs: update model structured output comments
* style: apply linting and formatting fixes
* fix: resolve type logic bug
* refactor: apply PR review feedback
* refactor: apply additional PR review feedback
* refactor: apply final PR review feedback
* fix: improve schema passing for LLMs without structured output
* feat: add PR comments and logger improvements
* docs: update README and add HTTP logging
* feat: improve HTTP logging
* feat: add summary chunking functionality
* fix: resolve title generation runtime issues
* refactor: apply self-review improvements
* style: apply linting and formatting
* feat: implement LiteLLM class structure
* style: apply linting and formatting fixes
* docs: env template model name fix
* chore: remove older litellm class
* chore: format
* refactor: simplify OpenAILLM
* refactor: OpenAILLM tokenizer
* refactor: self-review
* refactor: self-review
* refactor: self-review
* chore: format
* chore: remove LLM_USE_STRUCTURED_OUTPUT from envs
* chore: roll back migration lint changes
* chore: roll back migration lint changes
* fix: make summary llm configuration optional for the tests
* fix: missing f-string
* fix: tweak the prompt for summary title
* feat: try llamaindex for summarization
* fix: complete refactor of summary builder using llamaindex and structured output when possible
* fix: separate prompt as constant
* fix: typings
* fix: enhance prompt to prevent mentioning others subject while summarize one
* fix: various changes after self-review
* fix: from igor review
---------
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com >
2025-07-31 15:29:29 -06:00
5267ab2d37
feat: retake summary using NousResearch/Hermes-3-Llama-3.1-8B model ( #415 )
...
This feature a new modal endpoint, and a complete new way to build the
summary.
## SummaryBuilder
The summary builder is based on conversational model, where an exchange
between the model and the user is made. This allow more context
inclusion and a better respect of the rules.
It requires an endpoint with OpenAI-like completions endpoint
(/v1/chat/completions)
## vLLM Hermes3
Unlike previous deployment, this one use vLLM, which gives OpenAI-like
completions endpoint out of the box. It could also handle guided JSON
generation, so jsonformer is not needed. But, the model is quite good to
follow JSON schema if asked in the prompt.
## Conversion of long/short into summary builder
The builder is identifying participants, find key subjects, get a
summary for each, then get a quick recap.
The quick recap is used as a short_summary, while the markdown including
the quick recap + key subjects + summaries are used for the
long_summary.
This is why the nextjs component has to be updated, to correctly style
h1 and keep the new line of the markdown.
2024-09-14 02:28:38 +02:00
eba60b7de4
Merge branch 'main' into feat-api-speaker-reassignment
2023-12-15 11:14:19 +01:00
f7f67521fc
server: try reconcile both tests webrtc and upload with celery worker
2023-12-13 11:25:46 +01:00
6f3d7df507
server: add API to reassign speakers, and get topics with words
2023-12-06 16:41:18 +01:00
99b973f36f
server: fix tests
2023-11-22 14:41:40 +01:00
5ffa931822
server: update backend tests results (rpc does not work with chords)
2023-11-22 14:41:40 +01:00
4da890b95f
server: add dummy diarization and fixes instanciation
2023-11-02 17:39:21 +01:00
07c4d080c2
server: refactor with diarization, logic works
2023-11-02 17:39:21 +01:00
projects-g
1d92d43fe0
New summary ( #283 )
...
* handover final summary to Zephyr deployment
* fix display error
* push new summary feature
* fix failing test case
* Added markdown support for final summary
* update UI render issue
* retain sentence tokenizer call
---------
Co-authored-by: Koper <andreas@monadical.com >
2023-10-13 22:53:29 +05:30
projects-g
6a43297309
Translation enhancements ( #247 )
2023-09-26 19:49:54 +05:30
Gokul Mohanarangan
0b00881ce4
update tests: LLM mock to return LLM TITLE for all cases
2023-09-25 10:22:41 +05:30
projects-g
9fe261406c
Feature additions ( #210 )
...
* initial
* add LLM features
* update LLM logic
* update llm functions: change control flow
* add generation config
* update return types
* update processors and tests
* update rtc_offer
* revert new title processor change
* fix unit tests
* add comments and fix HTTP 500
* adjust prompt
* test with reflector app
* revert new event for final title
* update
* move onus onto processors
* move onus onto processors
* stash
* add provision for gen config
* dynamically pack the LLM input using context length
* tune final summary params
* update consolidated class structures
* update consolidated class structures
* update precommit
* add broadcast processors
* working baseline
* Organize LLMParams
* minor fixes
* minor fixes
* minor fixes
* fix unit tests
* fix unit tests
* fix unit tests
* update tests
* update tests
* edit pipeline response events
* update summary return types
* configure tests
* alembic db migration
* change LLM response flow
* edit main llm functions
* edit main llm functions
* change llm name and gen cf
* Update transcript_topic_detector.py
* PR review comments
* checkpoint before db event migration
* update DB migration of past events
* update DB migration of past events
* edit LLM classes
* Delete unwanted file
* remove List typing
* remove List typing
* update oobabooga API call
* topic enhancements
* update UI event handling
* move ensure_casing to llm base
* update tests
* update tests
2023-09-13 11:26:08 +05:30
e12f9afe7b
server: implement user authentication (none by default)
2023-08-16 17:24:05 +02:00
