selfhostyourtech/reflector
2
0
Fork 0
mirror of https://github.com/Monadical-SAS/reflector.git synced 2025-12-20 20:29:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: retake summary using NousResearch/Hermes-3-Llama-3.1-8B model (#415)

Browse Source 
This feature a new modal endpoint, and a complete new way to build the
summary.

## SummaryBuilder

The summary builder is based on conversational model, where an exchange
between the model and the user is made. This allow more context
inclusion and a better respect of the rules.

It requires an endpoint with OpenAI-like completions endpoint
(/v1/chat/completions)

## vLLM Hermes3

Unlike previous deployment, this one use vLLM, which gives OpenAI-like
completions endpoint out of the box. It could also handle guided JSON
generation, so jsonformer is not needed. But, the model is quite good to
follow JSON schema if asked in the prompt.

## Conversion of long/short into summary builder

The builder is identifying participants, find key subjects, get a
summary for each, then get a quick recap.

The quick recap is used as a short_summary, while the markdown including
the quick recap + key subjects + summaries are used for the
long_summary.

This is why the nextjs component has to be updated, to correctly style
h1 and keep the new line of the markdown.
This commit is contained in:
Mathieu Virbel
2024-09-14 02:28:38 +02:00
committed by GitHub
parent 6c4eac04c1
commit 5267ab2d37
20 changed files with 1383 additions and 238 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
server/tests/test_transcripts_rtc_ws.py
View File

@@ -72,7 +72,6 @@ async def test_transcript_rtc_and_websocket(
ensure_casing,
nltk,
appserver,
sentence_tokenize,
):
# goal: start the server, exchange RTC, receive websocket events
# because of that, we need to start the server in a thread
@@ -176,7 +175,7 @@ async def test_transcript_rtc_and_websocket(
assert "FINAL_LONG_SUMMARY" in eventnames
ev = events[eventnames.index("FINAL_LONG_SUMMARY")]
assert ev["data"]["long_summary"] == "* LLM LONG SUMMARY \n"
assert ev["data"]["long_summary"] == "LLM LONG SUMMARY"
assert "FINAL_SHORT_SUMMARY" in eventnames
ev = events[eventnames.index("FINAL_SHORT_SUMMARY")]
@@ -230,7 +229,6 @@ async def test_transcript_rtc_and_websocket_and_fr(
ensure_casing,
nltk,
appserver,
sentence_tokenize,
):
# goal: start the server, exchange RTC, receive websocket events
# because of that, we need to start the server in a thread
@@ -343,7 +341,7 @@ async def test_transcript_rtc_and_websocket_and_fr(
assert "FINAL_LONG_SUMMARY" in eventnames
ev = events[eventnames.index("FINAL_LONG_SUMMARY")]
assert ev["data"]["long_summary"] == "* LLM LONG SUMMARY \n"
assert ev["data"]["long_summary"] == "LLM LONG SUMMARY"
assert "FINAL_SHORT_SUMMARY" in eventnames
ev = events[eventnames.index("FINAL_SHORT_SUMMARY")]