RAG Chatbot for Markdown Files

This guide demonstrates how to build a Retrieval-Augmented Generation (RAG) chatbot. The bot loads a directory of Markdown (.md) files, splits them into chunks, creates a searchable vector index, and uses an LLM to answer questions based on that specific data. This is perfect for creating a bot that answers questions from your documentation or FAQ files.

Modules Used:

langchain: The framework for building LLM applications.
langchain-openai: Integration for OpenAI models and embeddings.
langchain-chroma: Integration for the Chroma vector database.
python-dotenv: To manage API keys.

Prerequisites

OpenAI API Key: You need an API key from OpenAI.
Data: A folder containing .md files (e.g., docs/ or notes/).

Installation

pip install langchain langchain-community langchain-openai langchain-chroma python-dotenv

Setup

Create a .env file:

OPENAI_API_KEY=sk-your-key-here

The Code

Save this as doc_bot.py.

import os
import argparse
import sys
from dotenv import load_dotenv

# LangChain Imports
from langchain_community.document_loaders import DirectoryLoader, TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_chroma import Chroma
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate

# 1. Load Config
load_dotenv()
if not os.getenv("OPENAI_API_KEY"):
    print("Error: OPENAI_API_KEY not found in .env")
    sys.exit(1)

def create_knowledge_base(directory):
    print(f"Loading .md files from '{directory}'...")

    # Load documents
    # glob="**/*.md" finds all markdown files recursively
    loader = DirectoryLoader(directory, glob="**/*.md", loader_cls=TextLoader)
    docs = loader.load()

    if not docs:
        print("No markdown files found.")
        return None

    print(f"Loaded {len(docs)} documents. Splitting text...")

    # Split documents into chunks for embedding
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
    splits = text_splitter.split_documents(docs)

    print(f"Created {len(splits)} chunks. Building vector store...")

    # Create Vector Store (In-memory for this example)
    vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings())
    return vectorstore

def start_chat(vectorstore):
    # Initialize LLM
    llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)

    # Create Retrieval Chain
    # This prompt tells the LLM to use the context provided
    prompt = ChatPromptTemplate.from_template("""
    Answer the following question based only on the provided context:

    <context>
    {context}
    </context>

    Question: {input}
    """)

    document_chain = create_stuff_documents_chain(llm, prompt)
    retriever = vectorstore.as_retriever()
    retrieval_chain = create_retrieval_chain(retriever, document_chain)

    print("\n--- Doc Bot Ready (Type 'quit' to exit) ---")
    while True:
        query = input("\nQuestion: ")
        if query.lower() in ["quit", "exit"]:
            break

        response = retrieval_chain.invoke({"input": query})
        print(f"\nAnswer: {response['answer']}")

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="RAG Chatbot for Markdown Files")
    parser.add_argument("directory", help="Directory containing .md files")

    args = parser.parse_args()

    vs = create_knowledge_base(args.directory)
    if vs:
        start_chat(vs)

Usage

Prepare Data: Ensure you have a folder (e.g., my_docs) with some Markdown files inside.
Run the Bot:
```
python doc_bot.py ./my_docs
```
Ask Questions: The bot will answer based only on the information found in your Markdown files.

programming/python/python

Created: March 17, 2026 • Last Modified: January 28, 2026

Docs Login