llm-distillery
Use LLMs to distill large texts down to a manageable size by utilizing a map-reduce approach. This ensures that the text fits within a specified token limit, which is crucial when interfacing with Large Language Models (LLMs) in downstreams tasks.
An npm package that uses LLMs to distill large texts down to a manageable size with a map-reduce approach, ensuring text fits within a specified token limit before downstream LLM tasks. It reduces text size based on token count without losing the essence of the content, using the semantic-chunking library to split text into chunks that are then summarized. Parameters such as target token size, API base URL, chunking thresholds, model, stop tokens, and maximum distillation loops are configurable. It works with any OpenAI API compatible endpoint (such as together.ai) and defaults to a Llama 3 model.
Getting Started
Prerequisites
- Node.js installed on your system.
- An API key for running inference of OpenAI API compatible LLM models (together.ai, etc.).
Installation
Add this lib to your code page via npm install
npm install llm-distilleryRelated \\
-
chunk-match
A NodeJS library that semantically chunks text and matches it against a user query using cosine similarity for precise and relevant text retrieval.
-
embedding-utils
Vector math, similarity search, ANN indexing, clustering, async pipelines, evaluation metrics, and multi-provider embedding generation -- zero dependencies,…
-
extract-topics
Use LDA (Latent Dirichlet Allocation) to extract topics from text