Agents
🚀 Start Building Multi-Model Agents Locally on a Laptop 🚀
What is a SLIM?
SLIMs are Structured Language Instruction Models, which are small, specialized 1-3B parameter LLMs, finetuned to generate structured outputs (Python dictionaries and lists, JSON and SQL) that can be handled programmatically, and stacked together in multi-step, multi-model Agent workflows - all running on a local CPU.
New SLIMS Just released - check out slim-extract, slim-summarize, slim-xsum, slim-sa-ner, slim-boolean and slim-tags-3b
Check out the new examples below marked with ⭐
🔥🔥🔥 Web Services & Function Calls (code) 🔥🔥🔥
Check out the Intro videos
SLIM Intro Video
There are 16 SLIM models, each delivered in two packages - a Pytorch/Huggingface FP16 model, and a
quantized “tool” designed for fast inference on a CPU, using LLMWare’s embedded GGUF inference engine. In most cases, we would recommend that you start with the “tools” version of the models.
Getting Started
We have several ready-to-run examples in this repository:
| Example | Detail | |
|---|---|---|
| 1. Getting Started with SLIM Models (code / video) | Install the models and run hello world tests to see the models in action. | |
| 2. Getting Started with Function-Calling Agent (code / video) | Generate a Structured Report with LLMfx | |
| 3. Multi-step Complex Analysis with Agent (code / video) | Delivering Complex Research Analysis with SLIM Agents | |
| 4. Document Clustering (code) | Multi-faceted automated document analysis with Topics, Tags and NER | |
| 5. Two-Step NER Retrieval (code) | Using NER to extract name, and then using as basis for retrieval. | |
| 6. Using Sentiment Analysis (code) | Using sentiment analysis on earnings transcripts and a ‘if…then’ condition | |
| 7. Text2SQL - Intro (code) | Getting Started with SLIM-SQL-TOOL and Basic Text2SQL Inference | |
| 8. Text2SQL - E2E (code) | End-to-End Natural Langugage Query to SQL DB Query | |
| 9. Text2SQL - MultiStep (code) | Extract a customer name using NER and use in a Text2SQL query | |
| 10. ⭐ Web Services & Function Calls (code) | Generate 30 key financial analysis with SLIM function calls and web services | |
| 11. ⭐ Yes-No Questions with Explanations (code) | Analyze earnings releases with SLIM Boolean | |
| 12. ⭐ Extracting Revenue Growth (code) | Extract revenue growth from earnings releases with SLIM Extract | |
| 13. ⭐ Summary as a Function Call (code) | Simple Summarization as a Function Call with List Length Parameters | |
| 14. ⭐ Handling Not Found Extracts (code) | Multi-step Lookup strategy and handling not-found answers | |
| 15. ⭐ Extract + Lookup (code) | Extract Named Entity information and use for lookups with SLIM Extract | |
| 16. ⭐ Headline/Title as XSUM Function Call (code) | eXtreme Summarization (XSUM) with SLIM XSUM |
For information on all of the SLIM models, check out LLMWare SLIM Model Collection.
Models List
If you would like more information about any of the SLIM models, please check out their model card:
- extract - extract custom keys - slim-extract & slim-extract-tool
- summary - summarize function call - slim-summary & slim-summary-tool
- xsum - title/headline function call - slim-xsum & slim-xsum-tool
- ner - extract named entities - slim-ner & slim-ner-tool
- sentiment - evaluate sentiment - slim-sentiment & slim-sentiment-tool
- topics - generate topic - slim-topics & slim-topics-tool
- sa-ner - combo model (sentiment + named entities) - slim-sa-ner & slim-sa-ner-tool
- boolean - provides a yes/no output with explanation - slim-boolean & slim-boolean-tool
- ratings - apply 1 (low) - 5 (high) rating - slim-ratings & slim-ratings-tool
- emotions - assess emotions - slim-emotions & slim-emotions-tool
- tags - auto-generate list of tags - slim-tags & slim-tags-tool
- tags-3b - enhanced auto-generation tagging model - slim-tags-3b & slim-tags-3b-tool
- intent - identify intent - slim-intent & slim-intent-tool
- category - high-level category - slim-category & slim-category-tool
- nli - assess if evidence supports conclusion - slim-nli & slim-nli-tool
- sql - convert text into sql - slim-sql & slim-sql-tool
You may also want to check out these quantized ‘answer’ tools, which work well in conjunction with SLIMs for question-answer and summarization:
- bling-stablelm-3b-tool - 3b quantized RAG model - bling-stablelm-3b-gguf
- bling-answer-tool - 1b quantized RAG model - bling-answer-tool
- dragon-yi-answer-tool - 6b quantized RAG model - dragon-yi-answer-tool
- dragon-mistral-answer-tool - 7b quantized RAG model - dragon-mistral-answer-tool
- dragon-llama-answer-tool - 7b quantized RAG model - dragon-llama-answer-tool
Set up
No special setup for SLIMs is required other than to install llmware >=0.2.6, e.g., pip3 install llmware.
Platforms:
- Mac M1, Mac x86, Windows, Linux (Ubuntu 22 preferred, supported on Ubuntu 20 +)
- RAM: 16 GB minimum
- Python 3.9, 3.10, 3.11 (note: not supported on 3.12 yet)
- llmware >= 0.2.6 version