Preventing Out of Context Information
Learn how to prevent out of context information from being generated by your AI models.
The Out-of-Context Problem in RAG Systems
Picture this scenario: You’ve built a RAG system to answer questions about famous landmarks using your carefully curated knowledge base. A user asks, “When was the Eiffel Tower completed?” Your system retrieves a relevant document:
The response comes back:
At first glance, this might seem like a helpful response. It’s detailed, informative, and answers the user’s question. There’s just one problem: most of this information isn’t from your knowledge base. The model has ventured beyond the retrieved context, drawing from its pre-trained knowledge to provide what it thinks is a helpful answer.
This is the out-of-context problem in RAG systems – when a language model generates information not found in the retrieved documents. It’s one of the most challenging issues in RAG implementations, often referred to as “closed-domain hallucination.”
Understanding the Challenge
The root of this problem lies in how modern language models work. These models are trained on vast amounts of data and retain this knowledge. When asked a question, they naturally try to be helpful by combining:
- Information from the provided context
- Their pre-existing knowledge
- Patterns they’ve learned from similar questions
In our Eiffel Tower example, the model:
- Used the context correctly (location and designer)
- Added the completion date (1889) from its training data
- Included visitor statistics it “knew” from pre-training
While this additional information might be factually correct, it creates several problems:
- Users can’t verify the source of this information
- The response mixes verified knowledge base facts with external information
- There’s no distinction between what came from our documents and what didn’t
This behavior particularly impacts RAG systems because they’re specifically designed to provide information from a controlled set of documents. When the model starts adding external information, it undermines the entire purpose of having a curated knowledge base.
The Path to a Solution
The key to solving this problem lies in understanding that language models will naturally try to be helpful by providing complete answers. They need explicit constraints and clear instructions to override this behavior. Let’s look at how we can transform our example to prevent out-of-context information.
First, here’s how we typically implement a RAG system with minimal constraints:
This implementation has several weaknesses:
- The system message is too general
- The prompt doesn’t explicitly restrict external knowledge
- There’s no guidance on handling missing information
Here’s how we can strengthen our implementation:
Now, when we ask about the Eiffel Tower’s completion date, we get a different response:
This response might seem less helpful at first, but it’s actually much better because:
- It’s honest about what information is available
- It clearly indicates what the source documents tell us
- It maintains the integrity of our knowledge base
The improvement comes from two key changes:
- A stronger system message that explicitly defines the model’s role and limitations
- A structured prompt that:
- Uses clear, direct language (“STRICTLY”, “DO NOT”)
- Provides explicit instructions for handling missing information
- Reinforces the importance of staying within the provided context
Building a Complete Solution
To implement this approach in your own RAG system, you’ll need several components working together. Let’s walk through a complete implementation that demonstrates both the problem and its solution.
Setting Up the Environment
First, let’s set up our environment with the necessary imports and configurations:
This setup includes:
- Loading environment variables for API keys
- Setting up Galileo logging for tracking operations
- Creating an OpenAI client for model interactions
Understanding the Document Retriever
The document retriever is designed to demonstrate how incomplete context can lead to out-of-context information:
Key points about the retriever:
- It simulates real-world document retrieval with intentionally incomplete information
- Uses predefined contexts to demonstrate the out-of-context problem
- Includes metadata for tracking document sources and relevance
Demonstrating the Problem
Let’s look at how a weak prompt can lead to out-of-context information:
Problems with this approach:
- The weak prompt doesn’t explicitly constrain the model
- The system message is too generic
- No explicit instruction to avoid using external knowledge
Implementing the Solution
Now, let’s see how to prevent out-of-context information with a stronger prompt:
Key improvements:
- Explicit instruction to use only provided context
- Clear directive to acknowledge when information is missing
- Stronger system message that reinforces context adherence
Running the Interactive Demo
The main function provides an interactive way to test and compare both approaches:
Using this code, you can:
- Test predefined queries that highlight the problem
- Compare responses from both approaches
- See the effectiveness of context constraints
Analyzing the Results
When running the demo, you’ll notice:
- Unconstrained Responses: May include information not present in the context
- Constrained Responses: Strictly adhere to provided information
- Completeness vs. Accuracy: Trade-off between complete answers and factual accuracy
Here’s an example comparison:
Query: “When was the Eiffel Tower completed?”
Unconstrained Response:
Constrained Response:
The constrained response demonstrates better adherence to the available context, even though it provides less information.
Best Practices and Recommendations
To prevent out-of-context information in your RAG system:
-
Strong Prompting:
- Be explicit about using only provided context
- Include clear instructions for handling missing information
- Use system messages that reinforce context adherence
-
Context Management:
- Ensure retrieved documents are relevant and complete
- Include metadata for tracking document sources
- Monitor and log context utilization
-
Response Validation:
- Compare responses against provided context
- Track and measure context adherence
- Use Galileo metrics to monitor performance
-
User Experience:
- Clearly communicate when information is limited
- Provide transparent source attribution
- Balance completeness with accuracy