Commit Graph

178 Commits

Author SHA1 Message Date
00c06b7971 server: use redis pubsub for interprocess websocket communication 2023-11-02 17:39:21 +01:00
8bebb2a769 server: start moving to an external celery task 2023-11-02 17:39:21 +01:00
f4cffc0e66 server: add tests on segmentation and fix issue with speaker 2023-11-02 17:39:21 +01:00
21e408b323 server: include transcripts words in database, but keep back compatible api 2023-11-02 17:39:21 +01:00
00eb9bbf3c server: improve split algorithm 2023-11-02 17:39:21 +01:00
b323254376 server: move out profanity filter to transcript, and implement segmentation 2023-11-02 17:39:21 +01:00
726efbc97a server: allow extra environment variable to ease switch between branch
suggested by Andreas
2023-11-02 12:24:52 +01:00
fe6d096f6f server: do not filter empty and recording for now 2023-11-02 12:24:52 +01:00
b8df3cde8c server: ensure loading bad topic won't be problematic 2023-11-02 12:24:52 +01:00
a3e9683f1e server: update get_all to filter empty and unfinished transcripts 2023-11-02 12:24:52 +01:00
86f4de7958 server: allow CORS credentials 2023-10-18 12:41:15 +02:00
c55f038ed4 server: allow to configure CORS_ORIGIN 2023-10-17 20:38:44 +02:00
Gokul Mohanarangan
c1a9005ec3 update buller condition 2023-10-14 18:55:40 +05:30
Gokul Mohanarangan
79fa537c35 update return format 2023-10-14 18:08:16 +05:30
Gokul Mohanarangan
894c989d60 update language codes 2023-10-14 17:35:30 +05:30
Sara
90c6824f52 replace two letter codes with three letter codes 2023-10-13 23:36:02 +02:00
projects-g
1d92d43fe0 New summary (#283)
* handover final summary to Zephyr deployment

* fix display error

* push new summary feature

* fix failing test case

* Added markdown support for final summary

* update UI render issue

* retain sentence tokenizer call

---------

Co-authored-by: Koper <andreas@monadical.com>
2023-10-13 22:53:29 +05:30
Sara
21473b584f Merge branch 'main' of github.com:Monadical-SAS/reflector into sara/recorder-memory 2023-10-13 19:10:55 +02:00
projects-g
628c69f81c Separate out transcription and translation into own Modal deployments (#268)
* abstract transcript/translate into separate GPU apps

* update app names

* update transformers library version

* update env.example file
2023-10-13 22:01:21 +05:30
Koper
1a7da94cae Fix MP3 download python error 2023-10-13 10:37:45 +01:00
Sara
fb7a7602f3 waveform is a list 2023-10-13 11:34:22 +02:00
Koper
abf9dbcaf1 Keep range_requests_response 2023-10-13 10:17:36 +01:00
Sara
a1df512603 Merge branch 'main' of github.com:Monadical-SAS/reflector into sara/recorder-memory 2023-10-13 11:05:02 +02:00
Koper
33d82bc9af Force MP3 download 2023-10-13 10:03:13 +01:00
Sara
006ab10b44 use numpy in waveform calculation 2023-10-13 10:51:55 +02:00
35af25d4e8 update export tools for danswer to match the plugin 2023-10-12 10:48:42 +02:00
953e51ef9f export danswer as text file 2023-10-12 10:48:42 +02:00
3f5133e419 server: add a tool to export a reflector sqlite3 database to csv 2023-10-12 10:48:42 +02:00
projects-g
bbe63ad407 Fix extra space between some tokens (punctuations) (#267)
* ensure uptime for reflector.media

* remove extra space before punct

* update detokenizer method

* create detokenizer property

* merge conflict
2023-10-12 10:42:19 +05:30
Sara
2cf61b191f fix waveform generation 2023-10-09 16:00:01 +02:00
47f7e1836e server: remove warmup methods everywhere 2023-10-06 13:59:17 -04:00
projects-g
e78bcc9190 Scaleai Translation (#258)
* hotfix

* remove assert from translation

* review comments

* reflector.media change targetLang to en
2023-09-28 18:16:39 +05:30
6c2651c63e Default length to 750 2023-09-27 10:47:58 -04:00
projects-g
24aa9a74bd hotfix (#254) 2023-09-27 19:20:43 +05:30
projects-g
6a43297309 Translation enhancements (#247) 2023-09-26 19:49:54 +05:30
Gokul Mohanarangan
f56eaeb6cc dont delete censored words 2023-09-25 21:25:18 +05:30
projects-g
3a374ea593 Delete server/reflector/llm/llm_params_cod.py 2023-09-25 09:46:00 +05:30
Gokul Mohanarangan
117acfacae update comment 2023-09-25 09:43:02 +05:30
Gokul Mohanarangan
80fd5e6176 update llm params 2023-09-22 07:49:41 +05:30
Gokul Mohanarangan
009d52ea23 update casing and trimming 2023-09-22 07:29:01 +05:30
Gokul Mohanarangan
ab41ce90e8 add profanity filter, post-process topic/title 2023-09-21 11:12:00 +05:30
2b9eef6131 server: use mp3 as default for audio storage
Closes #223
2023-09-13 17:26:03 +02:00
fb93c55993 server: fix nltk download 2023-09-13 11:40:39 +02:00
Gokul Mohanarangan
ed83236145 remove cache dir 2023-09-13 14:41:38 +05:30
Gokul Mohanarangan
9a10eef789 add nltk lookup path 2023-09-13 14:13:31 +05:30
projects-g
9fe261406c Feature additions (#210)
* initial

* add LLM features

* update LLM logic

* update llm functions: change control flow

* add generation config

* update return types

* update processors and tests

* update rtc_offer

* revert new title processor change

* fix unit tests

* add comments and fix HTTP 500

* adjust prompt

* test with reflector app

* revert new event for final title

* update

* move onus onto processors

* move onus onto processors

* stash

* add provision for gen config

* dynamically pack the LLM input using context length

* tune final summary params

* update consolidated class structures

* update consolidated class structures

* update precommit

* add broadcast processors

* working baseline

* Organize LLMParams

* minor fixes

* minor fixes

* minor fixes

* fix unit tests

* fix unit tests

* fix unit tests

* update tests

* update tests

* edit pipeline response events

* update summary return types

* configure tests

* alembic db migration

* change LLM response flow

* edit main llm functions

* edit main llm functions

* change llm name and gen cf

* Update transcript_topic_detector.py

* PR review comments

* checkpoint before db event migration

* update DB migration of past events

* update DB migration of past events

* edit LLM classes

* Delete unwanted file

* remove List typing

* remove List typing

* update oobabooga API call

* topic enhancements

* update UI event handling

* move ensure_casing to llm base

* update tests

* update tests
2023-09-13 11:26:08 +05:30
a3edd7a7cd server: update metric for rtc_offer 2023-09-12 20:46:01 +02:00
60edca6366 server: add prometheus instrumentation 2023-09-12 13:11:13 +02:00
600f2ca370 server: add BroadcastProcessor tests 2023-08-31 14:48:12 +02:00
9ed26030a5 server: add Broadcast processor 2023-08-31 14:48:12 +02:00