📚 Local PDF RAG (Rust + Rig 0.30 + Ollama + Qdrant)

A Rust project that implements a fully local RAG (Retrieval-Augmented Generation) pipeline:

1. Index PDFs: extracts text per page (pdftotext), chunks it, generates embeddings with Ollama, and stores vectors in Qdrant.
2. Ask questions: retrieves the most relevant chunks from Qdrant and builds a prompt with evidence citations ([#]) for the LLM to answer.

---

🧩 High-level architecture

• PDFs → pdftotext (Poppler) → per-page text
• Chunking → overlapped chunks + safe truncation
• Embeddings → Ollama (e.g., nomic-embed-text) via Rig
• Vector DB → Qdrant (collection dimension = EMBED_DIMS)
• Ask → top-k search in Qdrant → prompt with context → LLM (e.g., llama3.2:latest)

---

🧰 Tech stack

• Rust (edition 2024)
• Rig Core 0.30 (LLM + embeddings client)
• Ollama (local LLM + embeddings)
• Qdrant (vector database)
• Poppler pdftotext (stable PDF text extraction on Windows)
• Clap + dotenvy (.env)
• tracing / tracing-subscriber (logging)

---

✅ Prerequisites

• Rust + Cargo
• Docker Desktop (for Qdrant)
• Ollama installed and running
• Poppler / pdftotext available in PATH

Quick checks:

pdftotext -v
ollama list
docker ps

Check Qdrant

REST (HTTP) — port 6333:

Invoke-RestMethod "http://localhost:6333/collections"

gRPC (HTTP/2) — port 6334 (connectivity only):

Test-NetConnection localhost -Port 6334

---

⚙️ Configuration (.env)

Create a .env file at the project root (next to Cargo.toml):

# --- Qdrant ---
# Important:
# - 6333 = REST (for Invoke-RestMethod)
# - 6334 = gRPC (used by qdrant-client in Rust)
QDRANT_URL=http://localhost:6334
QDRANT_COLLECTION=pdf_rag

# --- Ollama (Rig reads this directly) ---
OLLAMA_API_BASE_URL=http://localhost:11434
EMBED_MODEL=nomic-embed-text
EMBED_DIMS=768
LLM_MODEL=llama3.2:latest

# --- Local state ---
STATE_FILE=rag_state.json

# --- Logs ---
RUST_LOG=pdf_rag_rig=info,rig=warn,qdrant_client=warn

# --- Index (to keep the command minimal) ---
PDF_DIR=<Path to your PDF files>
CHUNK_WORDS=220
OVERLAP_WORDS=40
BATCH_SIZE=48
MAX_CHARS=1800

# --- (Optional) Make Ask shorter ---
# TOP_K=6
# PRINT_PROMPT=false

Note: OLLAMA_API_BASE_URL is the key variable — Rig uses it to connect to Ollama.

---

▶️ Start dependencies

1) Qdrant (Docker)

docker run -d --name qdrant -p 6333:6333 -p 6334:6334 qdrant/qdrant

2) Ollama

Make sure Ollama is running and the models exist:

ollama list

---

🚀 Run

A) Index PDFs

Minimal command (uses PDF_DIR, MAX_CHARS, etc. from .env):

cargo run --release -- index

If you want to pass the path via CLI (while keeping the rest from .env):

cargo run --release -- index --pdf-dir "<Path to your PDF files>"

B) Ask a question (RAG)

cargo run --release -- ask --question "What does Martin Fowler say about messaging integration?" --top-k 6

Optional: print the final prompt:

cargo run --release -- ask --question "..." --top-k 6 --print-prompt true

---

♻️ Incremental indexing (does not reindex everything)

Indexing is incremental via STATE_FILE (e.g., rag_state.json) which stores a sha256 per PDF:

• If a PDF did not change, it logs SKIP (no changes)
• If a PDF is new or changed, it deletes old points in Qdrant for that source_file and reindexes it

⚠️ If you change chunking parameters (CHUNK_WORDS, OVERLAP_WORDS, MAX_CHARS), the PDF hash does not change, so it may still skip.
To force a full reindex:

• delete rag_state.json, or
• set a new STATE_FILE, or
• drop the Qdrant collection and reindex

---

🧪 Quality checks

cargo fmt
cargo clippy --all-targets -- -D warnings
cargo test
cargo build --release

---

🗂️ Project structure

• src/chunk.rs — normalization + chunking + safe truncation
• src/cli.rs — CLI parsing + wiring (Qdrant + Ollama + RAG)
• src/config.rs — project defaults
• src/lib.rs — public modules
• src/main.rs — loads .env, initializes logs, runs the CLI
• src/ollama_utils.rs — Ollama helpers (if used in your repo)
• src/pdf.rs — find PDFs and extract per-page text via pdftotext
• src/qdrant_store.rs — Qdrant connection / collection / vector store
• src/rag.rs — indexing + ask (vector search + prompt)
• src/state.rs — incremental state (hash per file)

---

👤 Author

Victor Aguayo