A retrieval-augmented generation (RAG) system can retrieve the right documents and still produce an inaccurate answer. This is why hallucination detection is essential for production RAG systems. Language models may introduce unsupported details, misinterpret evidence, or confidently present claims that contradict retrieved sources. These failures make reliable answer verification critical for AI applications.
LongTracer is EnDevSols' open-source verification and observability toolkit for RAG applications and AI-powered assistants. It helps developers evaluate generated responses against supporting documents, identify potentially hallucinated claims, calculate trust scores, and inspect verification results. Instead of relying entirely on the language model's confidence, LongTracer provides an additional evidence-checking layer between generated answers and the applications that consume them.
LongTracer combines claim-level analysis, semantic evidence matching, natural language inference (NLI), and structured tracing to support more reliable AI applications. It can be used through its Python API or integrated with existing frameworks such as LangChain, LlamaIndex, and LangGraph.
This guide explores how LongTracer works, its verification architecture, hallucination detection workflow, practical Python examples, framework integrations, trace storage, and observability capabilities. You'll also learn where LongTracer fits into a production RAG pipeline and how to use its verification signals to improve answer reliability.
Why RAG Systems Need Hallucination Detection
Retrieval-augmented generation improves the grounding of language models by supplying relevant information from external documents. However, retrieving relevant passages does not guarantee that the generated response accurately represents those passages.
A RAG pipeline typically involves document ingestion, chunking, embedding, retrieval, context assembly, and answer generation. Errors introduced at any stage can affect the final response. Even when retrieval succeeds, the language model may combine unrelated facts, introduce unsupported details, or make claims that contradict its source material.
For example, imagine a knowledge base containing an official refund policy that allows customers to request refunds within 30 days. A RAG assistant retrieves the correct policy but tells a customer they have 60 days. The retrieval component has worked, but the final answer contains an unsupported claim.
This creates an important distinction between retrieval quality and answer faithfulness. Retrieval quality measures whether the system finds useful evidence. Answer faithfulness measures whether the generated claims are supported by that evidence.
LongTracer addresses the second problem by evaluating generated responses against their supplied sources. Its verification process helps identify unsupported or contradictory claims, providing developers with measurable signals they can use to investigate and improve answer reliability.
Common Failure Modes in RAG Systems
| Failure type | What goes wrong | Why verification matters |
|---|---|---|
| Unsupported claim | The model adds information absent from retrieved sources | Flags claims lacking evidence |
| Contradictory answer | The response conflicts with a source document | Identifies conflicting claims |
| Incorrect citation | A cited passage does not support the statement | Helps expose citation mismatches |
| Mixed evidence | The model combines unrelated facts into one claim | Highlights potentially misleading conclusions |
Detecting these failures is important because a fluent, confident answer is not necessarily a trustworthy answer. Production RAG systems need a way to evaluate the relationship between generated claims and the evidence used to produce them.
How LongTracer Detects Hallucinations: Verification Architecture
LongTracer evaluates generated answers by comparing individual claims against their supporting source documents. Instead of treating an entire response as a single block of text, it separates the answer into smaller claims, identifies relevant evidence, and estimates whether that evidence supports or contradicts each claim.
The verification pipeline combines semantic textual similarity (STS) with natural language inference (NLI), allowing LongTracer to evaluate both the relevance of evidence and its relationship to the generated answer.
The Four Stages of Verification
1. Claim Splitting
LongTracer breaks a generated response into individual sentences or claims. This allows each statement to be assessed independently rather than assigning one judgment to the entire answer.
2. Semantic Evidence Matching (STS)
A sentence-transformer bi-encoder,
all-MiniLM-L6-v2, compares each claim with available source passages. The goal is to identify the most semantically relevant evidence for further verification.Semantic similarity helps locate related information, but similarity alone does not establish factual support.
3. Natural Language Inference (NLI)
LongTracer uses the
nli-deberta-v3-xsmall cross-encoder to evaluate the relationship between a claim and its matched evidence.The NLI model classifies the relationship into three categories:
- Entailment: The evidence supports the claim.
- Contradiction: The evidence conflicts with the claim.
- Neutral: The evidence does not establish whether the claim is correct.
This distinction helps identify unsupported statements even when the retrieved passage is closely related to the generated answer.
4. Trust Scoring and Verdict
LongTracer aggregates claim-level verification results into a trust score between 0.0 and 1.0, a hallucination count, and an overall verification verdict.
These results give developers a measurable indication of answer grounding. They are automated assessments, not guarantees of factual correctness, so verification thresholds should be calibrated against representative evaluation data.
Verification Pipeline Summary
| Stage | Input | Processing | Output |
|---|---|---|---|
| Claim splitting | Generated response | Separate response into claims | Individual claims |
| STS matching | Claims and source passages | Semantic similarity matching | Relevant evidence |
| NLI verification | Claim and matched evidence | Entailment, contradiction, or neutral classification | Claim-level assessments |
| Trust scoring | Verification assessments | Aggregate results | Trust score, hallucination count, and verdict |
Why Combine STS and NLI?
STS and NLI solve different problems. STS identifies which evidence is relevant to a claim, while NLI evaluates whether that evidence actually supports the claim.
For example, the statements "The Eiffel Tower is located in Paris" and "The Eiffel Tower is located in Berlin" are semantically similar because both discuss the same landmark and its location. However, they contradict each other.
A similarity-only approach might consider these statements closely related. An NLI-based verification stage can identify the conflicting location claim.
By combining both techniques, LongTracer provides a more informative verification signal than semantic similarity alone.
Getting Started with LongTracer: Python SDK
LongTracer provides a Python API for verifying LLM-generated responses against supporting source documents. Its core verification interface accepts plain text, making it suitable for existing RAG systems without requiring a particular vector database, language model, or orchestration framework.

Installation
Install LongTracer using pip:
pip install longtracerLongTracer requires Python 3.11 or newer. The base package provides verification functionality, while optional integrations can be installed separately when needed.
Verify Your First AI-Generated Answer
Consider a RAG assistant that incorrectly claims the Eiffel Tower is located in Berlin, even though the retrieved document identifies Paris.
from longtracer import check
result = check(
"The Eiffel Tower is 330 meters tall and located in Berlin.",
[
"The Eiffel Tower is a landmark in Paris, France. "
"It is 330 meters tall."
]
)
print(result.verdict)
print(result.trust_score)
print(result.hallucination_count)LongTracer compares the generated response with the supplied evidence. The incorrect location claim contradicts the source, allowing the verification process to identify a potential hallucination.
The result exposes three useful outputs:
- verdict: The overall verification outcome, such as PASS or FAIL.
- trust_score: A numerical assessment of how well the generated claims are supported.
- hallucination_count: The number of claims identified as potentially hallucinated.
The exact score depends on the verification models and configuration. A trust score should be treated as an evidence-grounding signal, not a guarantee that an answer is factually correct.
Verify Multiple Responses
LongTracer also supports batch verification, which is useful for evaluating multiple answers in testing workflows.
from longtracer import check_batch
results = check_batch([
{
"response": "The Eiffel Tower is located in Berlin.",
"sources": ["The Eiffel Tower is located in Paris."]
},
{
"response": "Water boils at 100°C at standard atmospheric pressure.",
"sources": [
"At standard atmospheric pressure, "
"the boiling point of water is 100°C."
]
}
])
for result in results:
print(result.verdict, result.trust_score)Batch verification helps developers inspect several generated responses without manually invoking the verification function for each one.
Explicit Verification States and Quality Gates
Some responses cannot be meaningfully verified. For example, an empty response contains no assessable claims.
LongTracer provides a typed verification interface through
check_case(), which distinguishes the verification execution state from the resulting quality decision.from longtracer import check_case
case = check_case("", ["Some supporting document text."])
print(case.availability)
print(case.reason)
print(case.quality_gate)For an empty response, the interface reports that no assessable claims are available, identifies the empty response as the reason, and returns a failing quality gate rather than treating the absence of claims as a successful verification.
This distinction is important for production pipelines because an answer that cannot be evaluated should not automatically be considered trustworthy.
Where to Run Verification
LongTracer can be called after an LLM generates a response and before the response is returned to the application or user.
A typical integration follows this sequence:
User question → Document retrieval → LLM-generated answer → LongTracer verification → Application decision
Based on the verification result, an application can accept the answer, request another generation attempt, flag it for review, or return a response indicating that sufficient evidence is unavailable.
These application decisions are configured by the developer; a verification score alone does not guarantee safe or reliable behavior.
Integrating LongTracer with RAG Frameworks
LongTracer is designed to work with existing RAG applications rather than requiring developers to rebuild their pipelines. Its framework adapters make it possible to add verification and tracing to applications built with LangChain, LlamaIndex, LangGraph, and Haystack.
Each integration is available through an optional dependency, keeping the core installation lightweight.

LangChain Integration
LangChain applications use chains to connect retrieval, prompting, and language model execution. LongTracer provides an instrumentation helper that can attach verification to an existing chain.
Install the integration:
pip install "longtracer[langchain]"from longtracer import LongTracer, instrument_langchain
LongTracer.init(verbose=True)
# Attach verification to an existing LangChain chain
instrument_langchain(your_chain)Here,
your_chain represents an existing, configured LangChain chain. Once instrumented, its execution can be traced and the generated responses evaluated against supporting evidence.LlamaIndex Integration
LlamaIndex applications commonly use query engines to retrieve knowledge and generate answers. LongTracer provides an adapter for instrumenting these workflows.
pip install "longtracer[llamaindex]"from longtracer import LongTracer, instrument_llamaindex
LongTracer.init(verbose=True)
# Instrument an existing query engine
instrument_llamaindex(your_query_engine)The adapter connects LongTracer to the application's existing query-engine workflow, helping developers inspect generated responses and verification behavior.
LangGraph Agent Integration
LangGraph supports stateful agent workflows, where execution may involve multiple nodes, retrieval steps, and tool calls.
LongTracer provides an instrumentation helper for tracing these workflows through callbacks.
pip install "longtracer[langgraph]"from longtracer import instrument_langgraph
# graph and agent are existing configured objects
handler = instrument_langgraph(graph)
result = agent.invoke(
{"messages": [("user", "What is retrieval-augmented generation?")]},
config={"callbacks": [handler]}
)The callback-based approach makes it possible to observe agent execution while retaining the application's existing graph structure.
Agent tracing and answer verification serve complementary purposes: tracing explains what happened during execution, while verification evaluates the final response against its supporting evidence.
Haystack Integration
Haystack applications organize retrieval and generation into connected pipeline components. LongTracer provides a verification component that can receive generated replies and retrieved documents.
pip install "longtracer[haystack]"from longtracer.adapters.haystack_handler import LongTracerVerifier
# pipeline already contains retriever and generator components
pipeline.add_component("verifier", LongTracerVerifier())
pipeline.connect("generator.replies", "verifier.response")
pipeline.connect("retriever.documents", "verifier.documents")This allows the verification stage to receive the generated answer and the documents used to produce it.
The example assumes that the existing Haystack pipeline exposes compatible component names and output types.
Framework Integration Overview
| Framework | Integration method | Primary use |
|---|---|---|
| LangChain | instrument_langchain() | Chain verification and tracing |
| LlamaIndex | instrument_llamaindex() | Query-engine verification |
| LangGraph | instrument_langgraph() | Agent workflow tracing |
| Haystack | LongTracerVerifier | Pipeline verification component |
| Custom Python | check() or CitationVerifier | Framework-independent verification |
Framework-Independent Verification
LongTracer's core verification API operates on generated text and supporting source passages. This means applications using OpenAI, Anthropic, Ollama, or other model providers can use LongTracer without adopting a particular orchestration framework.
For custom pipelines, developers can call the verification API directly after retrieving source documents and generating an answer.
This separation allows teams to introduce hallucination detection incrementally without changing their underlying retrieval architecture.
LongTracer Observability and Production Monitoring
Detecting hallucinations is only one part of maintaining a reliable RAG system. Developers also need visibility into verification results, recurring failures, and changes in answer quality over time.
LongTracer provides built-in tracing, persistent storage, a web dashboard, and OpenTelemetry integration to help teams investigate how their AI applications behave.

Built-in Web Dashboard
LongTracer includes a local web dashboard for inspecting verification traces and monitoring RAG reliability metrics.
Start the dashboard using:
longtracer serveOpen the dashboard in your browser:
http://localhost:8000/dashboardThe dashboard helps developers inspect verification activity, trust scores, hallucination rates, and stored traces without manually reviewing individual application logs.
Trace Storage Backends
LongTracer supports multiple storage backends, allowing developers to choose between lightweight local tracing and persistent storage for larger applications.
| Backend | Primary use | Installation |
|---|---|---|
| SQLite | Local development and persistent tracing | Built-in |
| Memory | Testing and temporary traces | Built-in |
| MongoDB | Shared and distributed trace storage | longtracer[mongo] |
| PostgreSQL | Relational trace storage | longtracer[postgres] |
| Redis | Key-value trace storage | longtracer[redis] |
Initialize LongTracer with a storage backend:
from longtracer import LongTracer
LongTracer.init(
project_name="production-rag",
backend="sqlite"
)SQLite is the default backend and stores traces locally at
~/.longtracer/traces.db. In-memory storage does not persist across application restarts.For production environments, developers can configure MongoDB, PostgreSQL, or Redis according to their infrastructure requirements.
Multi-Project Tracing
Teams operating multiple AI applications need a way to distinguish verification results from different services.
LongTracer supports project-specific tracing, allowing applications to maintain identifiable execution histories.
from longtracer import LongTracer
LongTracer.init(
project_name="customer-support-rag",
backend="sqlite"
)
tracer = LongTracer.get_tracer("customer-support-rag")
tracer.start_root(
inputs={"query": "What is the refund policy?"}
)Project names make it easier to filter traces, investigate failures, and separate monitoring data across applications.
Inspecting Traces from the CLI
LongTracer provides command-line tools for viewing previously recorded traces.
# View recent traces
longtracer view
# Inspect the latest trace
longtracer view --last
# Filter traces by project
longtracer view --project customer-support-rag
# Export a specific trace as JSON
longtracer view --export <trace_id>
# Generate an HTML trace report
longtracer view --html <trace_id>These commands help developers inspect verification history and export execution records for debugging or analysis.
OpenTelemetry Integration
For applications that already use distributed tracing, LongTracer supports OpenTelemetry.
Install the optional integration:
pip install "longtracer[otel]"With OpenTelemetry export configured, LongTracer can emit verification spans containing attributes such as:
trust_score— the generated answer's evidence-grounding score.hallucination_count— the number of claims flagged as potentially hallucinated.verdict— the overall verification result.
These spans can be exported to compatible observability systems, including Grafana Tempo, Jaeger, Datadog, and Honeycomb.
LongTracer also provides a Grafana dashboard template for visualizing verification telemetry.
Alerts and Reliability Monitoring
LongTracer supports notifications through webhooks, Slack, Discord, and email when configured trust-score thresholds are crossed.
This helps teams identify potentially unreliable responses and investigate recurring verification failures.
For production use, alert thresholds should be calibrated against representative evaluation data. A low trust score is a useful investigation signal, but it does not independently prove that an answer is factually incorrect.
By combining verification, tracing, persistent storage, and monitoring, LongTracer helps developers move beyond isolated hallucination checks toward continuous RAG reliability assessment.
LongTracer Production Best Practices and Real-World Use Cases
Integrating hallucination detection into a RAG application requires more than running a verification function. Production systems need carefully selected thresholds, reliable evidence, performance monitoring, and clear decisions for handling uncertain answers.

Best Practices for Production RAG Verification
1. Verify against the actual retrieved evidence
LongTracer should evaluate responses using the source passages available during answer generation. This helps identify unsupported claims and makes verification results relevant to the application's actual retrieval workflow.
2. Calibrate trust-score thresholds
A single trust-score threshold is not suitable for every application. Evaluate LongTracer against representative, human-labelled examples and adjust thresholds based on acceptable false positives, false negatives, and business risk.
3. Handle verification failures explicitly
Applications should distinguish between supported answers, potentially unsupported claims, and responses that cannot be evaluated. Depending on the result, they can return an answer, request another generation attempt, ask for human review, or respond that sufficient evidence is unavailable.
4. Monitor performance and latency
Verification adds processing after answer generation. Measure verification latency, model inference costs, throughput, and resource consumption under realistic workloads. Consider asynchronous processing or selective verification where immediate checks are not required.
5. Protect sensitive information in traces
Generated responses and retrieved documents may contain confidential information. Configure storage permissions, retention policies, access controls, and redaction before persisting production traces.
Understanding LongTracer's Limitations
LongTracer provides automated evidence-based assessments, not absolute factual guarantees.
Its effectiveness depends on the quality of the supplied evidence, claim extraction, semantic matching, and natural language inference models.
Important limitations include:
- Incomplete evidence: Missing source passages can make a correct answer appear unsupported.
- Model uncertainty: Semantic and NLI models can misclassify complex or ambiguous claims.
- Compound claims: Statements containing multiple facts may require more granular evaluation.
- Domain-specific language: Specialized terminology may require additional testing and threshold calibration.
- Verification overhead: Processing large responses or many source passages can increase latency and compute costs.
For high-impact applications, automated verification should complement testing, monitoring, and appropriate human oversight.
Real-World Use Cases
| Use case | Verification objective |
|---|---|
| Customer support assistants | Identify responses that conflict with documented support policies |
| Enterprise knowledge assistants | Check generated answers against internal knowledge sources |
| Financial document Q&A | Highlight unsupported claims about figures, dates, and financial statements |
| Legal document research | Identify potential contradictions between generated summaries and supplied documents |
| AI-powered research assistants | Evaluate whether generated statements are supported by retrieved references |
| Multi-tenant RAG applications | Monitor answer reliability across independently configured assistants |
When Should You Use LongTracer?
LongTracer is particularly useful when an application generates answers from retrieved documents and developers need measurable evidence-grounding signals.
It can be introduced into an existing RAG pipeline without replacing the application's language model, vector database, or orchestration framework.
For teams moving from prototypes to production, combining claim-level verification with tracing and monitoring provides a practical foundation for investigating hallucinations and improving answer reliability over time.
Frequently Asked Questions About LongTracer
What is LongTracer?
LongTracer is an open-source Python toolkit for detecting potential hallucinations, verifying AI-generated claims against supporting documents, and tracing verification activity in retrieval-augmented generation (RAG) applications.
How does LongTracer detect hallucinations?
LongTracer separates generated responses into claims, matches each claim with relevant source evidence using semantic similarity, and applies natural language inference to assess whether the evidence supports, contradicts, or does not establish the claim.
Does LongTracer require LangChain or LangGraph?
No. LongTracer provides a framework-independent Python verification API. Developers can use it directly or integrate it with supported frameworks such as LangChain, LlamaIndex, LangGraph, and Haystack.
Can LongTracer work with different language models?
Yes. Since LongTracer evaluates generated text against supplied evidence, it can be used with responses produced by different language model providers, including OpenAI, Anthropic, and locally hosted models.
Does LongTracer guarantee hallucination-free answers?
No. LongTracer provides automated evidence-grounding assessments, not guarantees of factual correctness. Results depend on the quality of retrieved evidence and verification models. High-impact applications should combine automated checks with appropriate human oversight.
Is LongTracer free and open source?
LongTracer is available as an open-source project on GitHub. Developers can inspect the repository, installation instructions, and license terms before integrating it into their applications.
Get Started with LongTracer
LongTracer helps developers move beyond basic document retrieval by adding claim-level verification, evidence matching, trust scoring, and observability to RAG applications.
To explore the source code, installation instructions, configuration options, and integration examples, visit the LongTracer GitHub repository.
For developers building complete production RAG applications, LongTracer can also be used alongside the other independent tools in Long Suite, EnDevSols' open-source ecosystem covering document ingestion, RAG application development, agent runtime protection, answer verification, and retrieval regression testing.
By combining reliable retrieval with evidence-based verification and continuous monitoring, teams can investigate unsupported answers, measure reliability, and improve the quality of AI-generated responses over time.
