LLM Based RAG Solutions

Large Language Models (LLMs) comes with built in knowledge and capable of Content Manipulation (creation, summarization, paraphrasing, translation, etc), Question Answering, Inferring Conclusions, Learning Patterns, etc. Prior to LLMs, all above tasks were only possible by intelligent humans. But LLMs have made it possible for super fast computers to carry out these actions with high accuracy.

Retrieval Augmented Generation

Retrieval Augmented Generation [RAG] is a way to feed context-specific, accurate and timely external information to LLMs. When combine these two aspects (1. LLM capabilities and 1. knowledge specific to a domain/entity), many entities could develop solutions to cater their internal and external information needs at much efficient way compared to getting humans involved.

RAG Workflow

Why Businesses Need RAG Solutions Today

• Internal Knowledge Assistants
• Customer Support Automation
• Policy/Document Search
• Technical Troubleshooting Guides
• Compliance And Auditing Support
• Product Discovery & Recommendations

RAG Related Models & Tools

Embedding Models

Vector Databases

Rerankers

Eval Frameworks

LLM Models

Retrievers

LLM Orchestrators

RAG Services

LLM & RAG Options

RAG Fine Tuning

1. Query

   Enrich with domain specific knowledge. Create sub queries, Prompt step-back questions

2. Indexing

   Optimizing the creation of chunking indexes using techniques such as multi-chunking, step-wise indexing, or multi-way indexing

3. Retriever

   Supplement vector similarity search with Knowledge Graphs, Keyword Searches etc

4. Generator

   Adjusting and optimizing query and context to LLM friendly. Break down the task to sub tasks

5. RAG Pipeline

   Dynamically switching paths within the RAG pipeline to use agents, tools, and any other transformers for the specific user query

Probabilistic vs Deterministic

Software systems are usually deterministic, meaning, for a given set of inputs, they always produce same set of outputs. This is what we are used to but when we use LLMs, certain parts of the solution become probabilistic. Meaning, it will not always produce the same set of outputs for a given set of inputs. This introduces a great number of difficulties when developing solutions using LLMs.

This uncertainty even could lead to unprecedented project failures. A PoC and an MVP which gave promising results could miserably fail in a production setup. So it is important to understand this complexity and deal with it in a pragmatic way from the initial stage of the solution development. And this found to be a common pitfall for many RAG solutions.

Our Approach

• Full Traceability

  Extensive message and information logging is a key apart of our RAG solution design. It enable both end user and developers equally to use and improve the system.

• Continuous Quality Checks

  Due to the probabilistic nature of LLMs it is important to built in evaluation processes to the system even on production stage. We build these features into the system from the beginning.

• Edge Cases

  Knowing where solution fail or will produce responses which are not acceptable is a key measurement. We spend time to identify what are the edge cases and make sure to establish necessary guardrails.

• Limitations of LLMs and RAGs

  LLMs could easily surprise you with exceptional results. But such results could easily fade off with time. So it is vital to know limitations of LLMs and design the systems under those conditions.

• Living Organism

  We treat a RAG based solution as a “living organism” rather than a traditional “build and ship” type solutions. Accuracy of the system could change due to so many factors so we take care of the system by grooming and nurturing it over time.

• Deterministic vs. Probabilistic

  Our approach is to use deterministic options as much as possible, and use probabilistic options only when necessary when building solutions. This help us to define and guarantee the system capabilities in a more predictable manner.