Completed

AI Knowledge Base Chat

Upload PDFs, Word files and text documents, then ask questions about them in plain English. A FastAPI backend chunks and embeds each document with local Ollama models, stores the vectors in Qdrant and the metadata in SQL Server, and streams back answers grounded only in the retrieved passages.

PythonFastAPIOllamaQdrantSQL ServerLangChainReact 19TypeScriptTanStack Query

Source on GitHub ↗

The problem

Company knowledge lives in handbooks and policy PDFs that nobody reads end to end, and keyword search finds the file, not the answer. The goal: upload documents once, ask questions in plain English, and get answers drawn only from those documents — with local models, so nothing leaves the machine.

Engineering focus

Retrieval-augmented generationPrimer
Multi-format ingestion and chunking
Local embeddings with Ollama
Vector search in Qdrant
Token streaming to React
Upload validation

In use

The Documents page with the Upload a document dialog open: PDF, DOCX, or TXT, up to 25MB, a file picker and an Upload button — Uploads accept PDF, DOCX or TXT up to 25 MB. The same limits are enforced again on the server, which does not trust the browser's check.

The Documents table: AI USAGE HANDBOOK.txt with status processing and no chunk count yet, IT_Policies_Knowledge_Base.pdf completed with 17 chunks, and an Upload started toast — The upload returns straight away; ingestion runs in the background. The table polls every two seconds only while a document is still pending or processing — here the handbook is mid-ingest while the policies PDF finished at 17 chunks.

The Chat page: the question 'Is there any remote access policy available?' answered with a summary of the Remote Access and VPN Usage policy from page 4 of the IT Policies Knowledge Base — An answer drawn from the uploaded IT policies PDF, citing the page the passage came from.

The Chat page in dark mode: 'Tell me about the VPN usage.' answered with three points on approved VPN or zero-trust access, the ban on split-tunneling, and mandatory MFA — Dark theme. The answer streams in token by token and is rendered as Markdown once it arrives.

The chat on a phone-width screen: the VPN answer, then 'Tell me about the leave policies.' answered with 20 days of paid annual leave per year, and a bottom tab bar for Documents and Chat — On a phone the navigation moves to a bottom tab bar.

The Qdrant dashboard Collections page listing the kb_chat collection: status green, 104 points, 768-dimension vectors, cosine distance — The kb_chat collection in the Qdrant dashboard: one point per chunk, 768-dimension vectors (nomic-embed-text's output size), cosine distance.

A point in the kb_chat collection: its payload holds the chunk text, page 0, document AI USAGE HANDBOOK.txt and doc_id 3, with a 768-length default vector — Each point carries its chunk text, source document, page and doc_id. The doc_id is what lets a delete remove every one of a document's vectors with a single filter.

How it works

Background ingestion: POST /api/documents saves the file and returns 202 with status pending. A background task then loads it, splits it into 500-character chunks with 100 overlap, embeds each one and upserts it — moving the row through processing to completed, or failed with the error recorded.
Upload validation: Extension allow-list, a 25 MB cap enforced while the file streams to disk, a %PDF- magic-byte check for PDFs, and a sanitised, UUID-prefixed filename. Anything rejected mid-stream is deleted, not left half-written.
Grounded answers: The question is embedded, the five nearest chunks come back from Qdrant, and llama3.2 is told to answer only from that context. An empty index returns a fixed message without calling the model; an unreachable Ollama returns a 503.
Streaming: /api/chat/stream returns plain-text tokens. The client reads the response stream, batches tokens into one re-render per animation frame, and aborts any earlier request when a new question is sent.
Two stores, one delete: SQL Server keeps each document's metadata, status and chunk text; Qdrant keeps the vectors. Deleting a document removes its SQL rows, its Qdrant points (filtered by doc_id) and the stored file.

Architecture

AI Knowledge Base Chat

PythonFastAPIOllamaQdrantSQL ServerLangChainReact 19TypeScriptTanStack Query

Source on GitHub ↗

The problem

In use

How it works

Background ingestion

POST /api/documents saves the file and returns 202 with status pending. A background task then loads it, splits it into 500-character chunks with 100 overlap, embeds each one and upserts it — moving the row through processing to completed, or failed with the error recorded.

Upload validation

Extension allow-list, a 25 MB cap enforced while the file streams to disk, a %PDF- magic-byte check for PDFs, and a sanitised, UUID-prefixed filename. Anything rejected mid-stream is deleted, not left half-written.

Grounded answers

The question is embedded, the five nearest chunks come back from Qdrant, and llama3.2 is told to answer only from that context. An empty index returns a fixed message without calling the model; an unreachable Ollama returns a 503.

Streaming

/api/chat/stream returns plain-text tokens. The client reads the response stream, batches tokens into one re-render per animation frame, and aborts any earlier request when a new question is sent.

Two stores, one delete

SQL Server keeps each document's metadata, status and chunk text; Qdrant keeps the vectors. Deleting a document removes its SQL rows, its Qdrant points (filtered by doc_id) and the stored file.