Prompts - Introduction by Examples

We introduce llmware through self-contained examples.

Basic RAG Scenario - Invoice Processing

""" This example shows an end-to-end scenario for invoice processing that can be run locally and without a
database.  The example shows how to combine the use of parsing combined with prompts_with_sources to rapidly
iterate through a batch of invoices and ask a set of questions, and then save the full output to both
(1) .jsonl for integration into an upstream application/database and (2) to a CSV for human review in excel.

    note: the sample code pulls from a public repo to load the sample invoice documents the first time -
    please feel free to substitute with your own invoice documents (PDF/DOCX/PPTX/XLSX/CSV/TXT) if you prefer.

    this example does not require a database or embedding

    this example can be run locally on a laptop by setting 'run_on_cpu=True'
    if 'run_on_cpu==False", then please see the example 'launch_llmware_inference_server.py'
    to configure and set up a 'pop-up' GPU inference server in just a few minutes
"""

import os
import re

from llmware.prompts import Prompt, HumanInTheLoop
from llmware.configs import LLMWareConfig
from llmware.setup import Setup
from llmware.models import ModelCatalog


def invoice_processing(run_on_cpu=True):

    #   Step 1 - Pull down the sample files from S3 through the .load_sample_files() command
    #   --note: if you need to refresh the sample files, set 'over_write=True'
    print("update: Downloading Sample Files")

    sample_files_path = Setup().load_sample_files(over_write=False)
    invoices_path = os.path.join(sample_files_path, "Invoices")

    #   Step 2 - simple sample query list - each question will be asked to each invoice
    query_list = ["What is the total amount of the invoice?",
                  "What is the invoice number?",
                  "What are the names of the two parties?"]

    #   Step 3 - Load Model

    if run_on_cpu:

        #   load local bling model that can run on cpu/laptop

        #   note: bling-1b-0.1 is the *fastest* & *smallest*, but will make more errors than larger BLING models
        # model_name = "llmware/bling-1b-0.1"

        #   try the new bling-phi-3 quantized with gguf - most accurate
        model_name = 'bling-phi-3-gguf'
    else:

        #   use GPU-based inference server to process
        #  *** see the launch_llmware_inference_server.py example script to setup ***

        server_uri_string = "http://11.123.456.789:8088"    # insert your server_uri_string
        server_secret_key = "demo-test"
        ModelCatalog().setup_custom_llmware_inference_server(server_uri_string, secret_key=server_secret_key)
        model_name = "llmware-inference-server"

    #   attach inference server to prompt object
    prompter = Prompt().load_model(model_name)

    #   Step 4 - main loop thru folder of invoices

    for i, invoice in enumerate(os.listdir(invoices_path)):

        #   just in case (legacy on mac os file system - not needed on linux or windows)
        if invoice != ".DS_Store":

            print("\nAnalyzing invoice: ", str(i + 1), invoice)

            for question in query_list:

                #   Step 4A - parses the invoices in memory and attaches as a source to the Prompt
                source = prompter.add_source_document(invoices_path,invoice)

                #   Step 4B - executes the prompt on the LLM (with the loaded source)
                output = prompter.prompt_with_source(question,prompt_name="default_with_context")

                for i, response in enumerate(output):
                    print("LLM Response - ", question, " - ", re.sub("[\n]"," ", response["llm_response"]))

                prompter.clear_source_materials()

    # Save jsonl report with full transaction history to /prompt_history folder
    print("\nupdate: prompt state saved at: ", os.path.join(LLMWareConfig.get_prompt_path(),prompter.prompt_id))

    prompter.save_state()

    # Generate CSV report for easy Human review in Excel
    csv_output = HumanInTheLoop(prompter).export_current_interaction_to_csv()

    print("\nupdate: csv output for human review - ", csv_output)

    return 0


if __name__ == "__main__":

    invoice_processing(run_on_cpu=True)

Document Summarizer

""" This Example shows a packaged 'document_summarizer' prompt using the slim-summary-tool. It shows a variety of
techniques to summarize documents generally larger than a LLM context window, and how to assemble multiple source
batches from the document, as well as using a 'query' and 'topic' to focus on specific segments of the document. """

import os

from llmware.prompts import Prompt
from llmware.setup import Setup


def test_summarize_document(example="jd salinger"):

    # pull a sample document (or substitute a file_path and file_name of your own)
    sample_files_path = Setup().load_sample_files(over_write=False)

    topic = None
    query = None
    fp = None
    fn = None

    if example not in ["jd salinger", "employment terms", "just the comp", "un resolutions"]:
        print ("not found example")
        return []

    if example == "jd salinger":
        fp = os.path.join(sample_files_path, "SmallLibrary")
        fn = "Jd-Salinger-Biography.docx"
        topic = "jd salinger"
        query = None

    if example == "employment terms":
        fp = os.path.join(sample_files_path, "Agreements")
        fn = "Athena EXECUTIVE EMPLOYMENT AGREEMENT.pdf"
        topic = "executive compensation terms"
        query = None

    if example == "just the comp":
        fp = os.path.join(sample_files_path, "Agreements")
        fn = "Athena EXECUTIVE EMPLOYMENT AGREEMENT.pdf"
        topic = "executive compensation terms"
        query = "base salary"

    if example == "un resolutions":
        fp = os.path.join(sample_files_path, "SmallLibrary")
        fn = "N2126108.pdf"
        # fn = "N2137825.pdf"
        topic = "key points"
        query = None

    # optional parameters:  'query' - will select among blocks with the query term
    #                       'topic' - will pass a topic/issue as the parameter to the model to 'focus' the summary
    #                       'max_batch_cap' - caps the number of batches sent to the model
    #                       'text_only' - returns just the summary text aggregated

    kp = Prompt().summarize_document_fc(fp, fn, topic=topic, query=query, text_only=True, max_batch_cap=15)

    print(f"\nDocument summary completed - {len(kp)} Points")
    for i, points in enumerate(kp):
        print(i, points)

    return 0


if __name__ == "__main__":

    print(f"\nExample: Summarize Documents\n")

    #   4 examples - ["jd salinger", "employment terms", "just the comp", "un resolutions"]
    #   -- "jd salinger" - summarizes key points about jd salinger from short biography document
    #   -- "employment terms" - summarizes the executive compensation terms across 15 page document
    #   -- "just the comp" - queries to find subset of document and then summarizes the key terms
    #   -- "un resolutions" - summarizes the un resolutions document

    summary_direct = test_summarize_document(example="employment terms")

For more examples, see the [prompt examples]((https://www.github.com/llmware-ai/llmware/tree/main/examples/Prompts/) in the main repo.

Check back often - we are updating these examples regularly - and many of these examples have companion videos as well.

More information about the project - see main repository

About the project

llmware is © 2023-2024 by AI Bloks.

Contributing

Please first discuss any change you want to make publicly, for example on GitHub via raising an issue or starting a new discussion. You can also write an email or start a discussion on our Discrod channel. Read more about becoming a contributor in the GitHub repo.

Code of conduct

We welcome everyone into the llmware community. View our Code of Conduct in our GitHub repository.

llmware and AI Bloks

llmware is an open source project from AI Bloks - the company behind llmware. The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service. AI Bloks was founded by Namee Oberst and Darren Oberst in October 2022.

License

llmware is distributed by an Apache-2.0 license.

Thank you to the contributors of llmware!

• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 

---

• 

---