Prompt with Sources

Prompt with Sources: the easiest way to combine knowledge retrieval with a LLM inference, and provides several high-level useful methods to easily integrate a retrieval/query/parsing step into a prompt to be used as a source for running an inference on a model.

This is best illustrated with a simple example:

from llmware.prompts import Prompt

#   build a prompt and attach a model
prompter = Prompt().load_model("llmware/bling-tiny-llama-v0")

#   add_source_document method: accepts any supported document type, parses the file, and creates text chunks
#   if a query is passed, then it will run a quick in-memory filtering search against the text chunks
#   the text chunks are packaged into sources with all of the accompanying metadata from the file, and made 
#   available automatically in batches to be used in prompting -

source = prompter.add_source_document("/folder/to/one/doc/", "filename", query="fast query")

#   to run inference with 'prompt with sources' -> source will be automatically added to the prompt
responses = prompter.prompt_with_source("my query")

#   depending upon the size of the source (and batching relative to the model context window, there may be more than 
#   a single inference run, so unpack potentially multiple responses

for i, response in enumerate(responses):
    print("response: ", i, response)

FACT CHECKING

Using prompt_with_source also provides integrated fact-checking methods that use the packaged source information to validate key elements from the llm_response

from llmware.prompts import Prompt

prompter = Prompt().load_model("bling-answer-tool", temperature=0.0, sample=False)

# contract is parsed, text-chunked, and then filtered by "base salary'
source = prompter.add_source_document("/local/folder/path", "my_document.pdf", query="exact filter query")

# calling the LLM with 'source' information from the contract automatically packaged into the prompt
responses = prompter.prompt_with_source("my question to the document", prompt_name="default_with_context")
    
# run several fact checks

#   checks for numbers match
ev_numbers = prompter.evidence_check_numbers(responses)

#   looks for statistical overlap to identify potential sources for the llm response
ev_sources = prompter.evidence_check_sources(responses)

#   builds set of comparison stats between the llm_response and the sources
ev_stats = prompter.evidence_comparison_stats(responses)

#   identifies if a response is a "not found" response
z = prompter.classify_not_found_response(responses, parse_response=True, evidence_match=True,ask_the_model=False)

for r, response in enumerate(responses):
    print("LLM Response: ", response["llm_response"])
    print("Numbers: ",  ev_numbers[r]["fact_check"])
    print("Sources: ", ev_sources[r]["source_review"])
    print("Stats: ", ev_stats[r]["comparison_stats"])
    print("Not Found Check: ", z[r])

In addition to add_source_document, the Prompt class implements the following other methods to easily integrate sources into prompts:

Add Source - Query Results - Two Options

from llmware.prompts import Prompt
from llmware.retrieval import Query
from llmware.library import Library

#   build a prompt
prompter = Prompt().load_model("llmware/bling-tiny-llama-v0")

#   Option A - run query and then add query_results to the prompt
my_lib = Library().load_library("my_library")
results = Query(my_lib).query("my query")

source2 = prompter.add_source_query_results(results)

#   Option B - run a new query against a library and load directly into a prompt
source3 = prompter.add_source_new_query(my_lib, query="my new query", query_type="semantic", result_count=15)

Add Other Sources

from llmware.prompts import Prompt

#   build a prompt
prompter = Prompt().load_model("llmware/bling-tiny-llama-v0")

#   add wikipedia articles as a source
wiki_source = prompter.add_source_wikipedia("topic", article_count=5, query="filter among retrieved articles")  

#   add a website as a source
website_source = prompter.add_source_website("my_url", query="filter among website")

#   add an entire library (should be small, e.g., just a couple of documents)
source = prompter.add_source_library("my_library")

Need help or have questions?

Check out the llmware videos and GitHub repository.

Reach out to us on GitHub Discussions.

About the project

Contributing

Please first discuss any change you want to make publicly, for example on GitHub via raising an issue or starting a new discussion. You can also write an email or start a discussion on our Discrod channel. Read more about becoming a contributor in the GitHub repo.

Code of conduct

We welcome everyone into the llmware community. View our Code of Conduct in our GitHub repository.

`llmware` and AI Bloks

llmware is an open source project from AI Bloks - the company behind llmware. The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service. AI Bloks was founded by Namee Oberst and Darren Oberst in Oktober 2022.

License

llmware is distributed by an Apache-2.0 license.