queue-tha-llama

This is a web-based chat application that integrates Large Language Model (LLM) capabilities with Bull Queue, Redis, and Chroma. It handles concurrent chat sessions with advanced queue management, maintains robust client-server communication with heartbeat signals, utilizes a RAG model for chat memory, and smartly manages inactive clients and job cleanups for a seamless chat experience.

A web-based chat application that integrates Large Language Model (LLM) capabilities with Bull Queue, Redis, and Chroma. It handles concurrent chat sessions with queue management, maintains client-server communication with heartbeat signals, and uses a RAG model for chat memory. It also manages inactive clients and job cleanups. It can run against a local LLM via llama.cpp or a cloud API, uses Redis and Chroma vector storage in Docker containers, and supports embedding models such as all-MiniLM-L6-v2.