---

Platform Support

---

Platform Supported

• Python 3.9+ (note that we just added support for 3.12 starting in llmware version 0.2.12)

• System RAM: recommended 16 GB RAM minimum (to run most local models on CPU)

• OS Supported: Mac OS M1/M2/M3, Windows, Linux Ubuntu 20/22. We regularly build and test on Windows and Linux platforms with and without CUDA drivers.

• Deprecated OS: Linux Aarch64 (0.2.6) and Mac x86 (0.2.10) - most features of llmware should work on these platforms, but new features integrated since those versions will not be available. If you have a particular need to work on one of these platforms, please raise an Issue, and we can work with you to try to find a solution.

• Linux: we build to GLIBC 2.31+ - so Linux versions with older GLIBC drivers will generally not work (e.g., Ubuntu 18). To check the GLIBC version, you can use the command ldd --version. If it is 2.31 or any higher version, it should work.

---

---

Database

• LLMWare is an enterprise-grade data pipeline designed for persistent storage of key artifacts throughout the pipeline. We provide several options to parse ‘in-memory’ and write to jsonl files, but most of the functionality of LLMWare assumes that a persistent scalable data store will be used.

• There are three different types of data storage used in LLMWare:

  1. Text Collection database - all of the LLMWare parsers, by default, parse and text chunk unstructured content (and associated metadata) into one of three databases used for text collections, organized in Libraries - MongoDB, Postgres and SQLite.

  2. Vector database - for storing and retrieving semantic embedding vectors, LLMWare supports the following vector databases - Milvus, PG Vector / Postgres, Qdrant, ChromaDB, Redis, Neo4J, Lance DB, Mongo-Atlas, Pinecone and FAISS.

  3. SQL Tables database - for easily integrating table-based data into LLM workflows through the CustomTable class and for using in conjunction with a Text-2-SQL workflow - supported on Postgres and SQLite.

• Fast Start option: you can start using SQLite locally without any separate installation by setting LLMWareConfig.set_active_db("sqlite") as shown in configure_db_example. For vector embedding examples, you can use ChromaDB, LanceDB or FAISS - all of which provide no-install options - just start using.

• Install DB dependencies: we provide a number of Docker-Compose scripts which can be used, or follow install instructions provided by the database - generally easiest to install locally with Docker.

LLMWare File Storage

• llmware stores a variety of artifacts during its operation locally in the /llmware_data path, which can be found as follows:

from llmware.configs import LLMWareConfig
llmware_fp = LLMWareConfig().get_llmware_path()
print("llmware_data path: ", llmware_fp)

• to change the llmware path, we can change both the ‘home’ path, which is the main filepath, and the ‘llmware_data’ path name as follows:

from llmware.configs import LLMWareConfig

# changing the llmware home path - change home + llmware_path_name
LLMWareConfig().set_home("/my/new/local/home/path")
LLMWareConfig().set_llmware_path_name("llmware_data2")

# check the new llmware home path
llmware_fp = LLMWareConfig().get_llmware_path()
print("updated llmware path: ", llmware_fp)

---

---

Local Models

• LLMWare treats open source and locally deployed models as “first class citizens” with all classes, methods and examples designed to work first with smaller, specialized, locally-deployed models.
• By default, most models are pulled from public HuggingFace repositories, and cached locally. LLMWare will store all models locally at the /llmware_data/model_repo path, with all assets found in a folder tree with the models name.
• If a Pytorch model is pulled from HuggingFace, then it will appear in the default HuggingFace /.cache path.
• To view the local model path:

from llmware.configs import LLMWareConfig

model_fp = LLMWareConfig().get_model_repo_path()
print("model repo path: ", model_fp)

More information about the project - see main repository

About the project

llmware is © 2023-2024 by AI Bloks.

Contributing

Please first discuss any change you want to make publicly, for example on GitHub via raising an issue or starting a new discussion. You can also write an email or start a discussion on our Discrod channel. Read more about becoming a contributor in the GitHub repo.

Code of conduct

We welcome everyone into the llmware community. View our Code of Conduct in our GitHub repository.

llmware and AI Bloks

llmware is an open source project from AI Bloks - the company behind llmware. The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service. AI Bloks was founded by Namee Oberst and Darren Oberst in October 2022.

License

llmware is distributed by an Apache-2.0 license.

Thank you to the contributors of llmware!

• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 

---

• 

---