pieter/eveAI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
148 Commits 3 Branches 24 Tags
18074353399b647963ea64ef459e86c5450bc1f8
Commit Graph

25 Commits

Author SHA1 Message Date
Josako
1807435339 - Introduction of dynamic Retrievers & Specialists 
- Introduction of dynamic Processors
- Introduction of caching system
- Introduction of a better template manager
- Adaptation of ModelVariables to support dynamic Processors / Retrievers / Specialists
- Start adaptation of chat client
 2024-11-15 10:00:53 +01:00
Josako
3e644f1652 - Add Catalog Functionality 2024-10-15 18:14:57 +02:00
Josako
9f5f090f0c - License Usage Calculation realised 
- View License Usages
- Celery Beat container added
- First schedule in Celery Beat for calculating usage (hourly)
- repopack can now split for different components
- Various fixes as consequece of changing file_location / file_name ==> bucket_name / object_name
- Celery Routing / Queuing updated
 2024-10-11 16:33:36 +02:00
Josako
9782e31ae5 - Refined entitlements to work with MiB for both embeddings and storage 
- Improved DocumentVersion storage attributes to reflect Minio settings
- Added size to DocumentVersions to easily calculate usage
- License / LicenseTier forms and views added
 2024-10-07 14:17:44 +02:00
Josako
b700cfac64 - Improvements on audio processing to limit CPU and memory usage 
- Removed Portkey from the equation, and defined explicit monitoring using Langchain native code
- Optimization of Business Event logging
 2024-10-02 14:11:46 +02:00
Josako
d9cb00fcdc Business event tracing completed for both eveai_workers tasks and eveai_chat_workers tasks 2024-09-27 10:53:42 +02:00
Josako
ee1b0f1cfa Start log tracing to log business events. Storage in both database and logging-backend. 2024-09-25 15:39:25 +02:00
Josako
6cf660e622 - Adding a Tenant Type 
- Allow filtering on Tenant Types & searching for parts of Tenant names
- Implement health checks
- Start Prometheus monitoring (needs to be finalized)
- Refine audio_processor and srt_processor to reduce duplicate code and support for larger files
- Introduce repopack to reason in LLMs about the code
 2024-09-13 15:43:40 +02:00
Josako
914c265afe - Improvements on document uploads (accept other files than html-files when entering a URL) 
- Introduction of API-functionality (to be continued). Deduplication of document and url uploads between views and api.
- Improvements on document processing - introduction of processor classes to streamline document inputs
- Removed pure Youtube functionality, as Youtube retrieval of documents continuously changes. But added upload of srt, mp3, ogg and mp4
 2024-09-02 12:37:44 +02:00
Josako
122d1a18df - Allow for more complex and longer PDFs to be uploaded to Evie. First implmentation of a processor for specific file types. 
- Allow URLs to contain other information than just HTML information. It can alose refer to e.g. PDF-files.
 2024-08-27 07:05:56 +02:00
Josako
2ca006d82c Added excluded element classes to HTML parsing to allow for more complex document parsing 
Added chunking to conversion of HTML to markdown in case of large files
 2024-08-22 16:41:13 +02:00
Josako
ab38dd7540 - Improvements working with the cloud, minio, graylog and some first bugfixing 2024-08-13 09:04:19 +02:00
Josako
64cf8df3a9 - Improvements to enable deployment in the cloud, mainly changing file access to Minio 
- Improvements on RAG logging, and some debugging in that area
 2024-08-01 17:35:54 +02:00
Josako
908a2eaf7e - Improve annotation algorithm for Youtube (and others) 
- Patch Pytube
- improve OS deletion of files and writing of files
- Start working on Claude
- Improve template management
 2024-07-16 14:21:49 +02:00
Josako
ea0127b4b8 Improve algorithms for HTML and PDF processing 2024-07-08 15:20:45 +02:00
Josako
8e1dac0233 Youtube added - further checking required 2024-07-04 08:11:31 +02:00
Josako
be311c440b Improving chat functionality significantly throughout the application. 2024-06-12 11:07:18 +02:00
Josako
27b6de8734 Removing DocumentLanguage, as both System Context and User Context are to be defined on DocumentVersion level. 
Finetuning of embedding workers.
 2024-06-06 15:26:49 +02:00
Josako
61e1372dc8 Improvements to Document Interface and correcting embedding workers 2024-06-04 14:59:38 +02:00
Josako
6c2e99f467 Realise processing of HTML and improve both HTML & PDF processing giving new tenant information. 2024-05-13 17:18:38 +02:00
Josako
011bdce38d Prepare for html document validation (added wanted tags to tenant) 2024-05-12 21:58:42 +02:00
Josako
a4bf837d67 Start working on chunking en embedding task. Continu with embeddings. 2024-05-08 22:40:55 +02:00
Josako
cd5afa0408 Refactoring finished :-) 
eveai_workers now working (with errors ;-) )
Remote debugging now available
 2024-05-07 22:51:48 +02:00
Josako
131c609e68 Refactoring part 2 
Necessary changes to ensure correct working of eveai_app
 2024-05-06 23:07:45 +02:00
Josako
8e5ad5f312 Refactoring part 1 
Some changes for workers, but stopped due to refactoring
 2024-05-06 21:30:07 +02:00