LongTrainer: Open-Source RAG Chatbot Framework for Production AI

Why Production RAG Chatbots Need More Than Basic Retrieval

Common Challenges in Production RAG Systems

What Is LongTrainer?

Core Features of LongTrainer

• Multi-tenant chatbot management: Create multiple bots with independent configurations and sessions.
• Persistent conversation memory: Store and restore chat history using MongoDB.
• Document ingestion: Load documents and information from supported files, URLs, and external services.
• Vector search: Retrieve relevant document passages before generating answers.
• Streaming responses: Deliver generated output incrementally.
• AI agents and tools: Build tool-enabled assistants with LangGraph.
• Model customization: Configure language models, embeddings, prompts, and retrieval parameters per bot.
• Observability: Integrate with LongTracer for tracing and evidence-based response verification.

LongTrainer Architecture: From Documents to Intelligent Chatbots

1. Document Ingestion

2. Embeddings and Vector Search

3. Language Model Execution

4. Persistent Conversation Memory

5. Agent and Tool Execution

Getting Started with LongTrainer: Installation and Python SDK

Installation

pip install longtrainer

pip install "longtrainer[agent]"

pip install "longtrainer[tracer]"

Build Your First Document-Aware Chatbot

from longtrainer.trainer import LongTrainer

# Requires a configured OPENAI_API_KEY
# and a running MongoDB instance.

trainer = LongTrainer(
    mongo_endpoint="mongodb://localhost:27017/"
)

# Initialize an independent bot
bot_id = trainer.initialize_bot_id()

# Add a local document
trainer.add_document_from_path(
    "documents/company_policy.pdf",
    bot_id
)

# Build the chatbot
trainer.create_bot(bot_id)

# Start a conversation
chat_id = trainer.new_chat(bot_id)

# Ask a document-grounded question
answer, sources = trainer.get_response(
    "What is the company refund policy?",
    bot_id,
    chat_id
)

print(answer)

Understanding the Main SDK Methods

Multi-Tenant Chatbot Management and Persistent Memory

Per-Bot Configuration

from langchain_openai import ChatOpenAI, OpenAIEmbeddings

trainer.create_bot(
    bot_id,
    llm=ChatOpenAI(
        model="gpt-4o-mini",
        temperature=0.2
    ),
    embedding_model=OpenAIEmbeddings(
        model="text-embedding-3-small"
    ),
    num_k=5,
    prompt_template=(
        "You are a helpful document assistant. "
        "Use the following context: {context}"
    )
)

MongoDB Conversation Persistence

Streaming Responses for Real-Time AI Chatbots

Synchronous Streaming

for chunk in trainer.get_response(
    "Summarize the main findings",
    bot_id,
    chat_id,
    stream=True
):
    print(chunk, end="", flush=True)

Asynchronous Streaming

async for chunk in trainer.aget_response(
    "Explain the document",
    bot_id,
    chat_id
):
    print(chunk, end="", flush=True)

AI Agents and Tool Calling with LangGraph

RAG Mode vs. Agent Mode

Registering a Custom Tool

from langchain_core.tools import tool

@tool
def multiply(left: float, right: float) -> float:
    """Multiply two numbers."""
    return left * right

trainer.add_tool(multiply, bot_id)

trainer.create_bot(
    bot_id,
    agent_mode=True
)

chat_id = trainer.new_chat(bot_id)

response, sources = trainer.get_response(
    "What is 42 multiplied by 17?",
    bot_id,
    chat_id
)

print(response)

Named Agent Types in LongTrainer 1.4.0

Document Ingestion and Vector Database Integration

Supported Ingestion Sources

Retrieval Configuration

• num_k for controlling the number of retrieved documents.
• chunk_size for text splitting.
• chunk_overlap for preserving context between chunks.
• ensemble for enabling multi-query retrieval behavior.

Deploying LongTrainer Through CLI and REST API

Command-Line Setup

longtrainer init

longtrainer bot create --prompt "You are a helpful assistant."

longtrainer add-doc BOT_ID documents/manual.pdf

longtrainer chat BOT_ID

FastAPI Server

longtrainer serve

http://localhost:8000

http://localhost:8000/docs

LongTrainer Observability and LongTracer Integration

pip install "longtrainer[tracer]"

LongTrainer Production Best Practices and Real-World Use Cases

Best Practices for Production RAG

Real-World Use Cases

LongTrainer Limitations and Production Considerations

• Retrieval limitations: Missing or irrelevant source content can lead to weak answers.
• Model uncertainty: LLMs can generate unsupported statements even when relevant evidence exists.
• Infrastructure dependencies: MongoDB, model providers, and selected document loaders must be configured.
• Agent security: External tools can introduce operational and security risks.
• Context limitations: Persistent memory does not guarantee that all historical information reaches the model.
• Cost and latency: Embeddings, tool execution, retrieval, and model inference consume resources.
• Release compatibility: New features documented in the development repository may not yet exist in the published package.

LongTrainer and the Long Suite Ecosystem

Frequently Asked Questions About LongTrainer

What is LongTrainer?

Is LongTrainer built on LangChain?

Can LongTrainer create multiple chatbots?

Does LongTrainer support persistent chat memory?

Which vector databases does LongTrainer support?

Can LongTrainer stream AI responses?

Does LongTrainer support AI agents?

Is LongTrainer free and open source?

Does LongTrainer guarantee accurate answers?

Get Started with LongTrainer