A research group's papers, notes and internal documents, turned into a chatbot that answers with citations and drops into the group's own web page with a script tag. OuRAGboros is the service; Chatbot-Widget is the 12 KB front end. Both are open source under SLAC-ML, and the service runs on SLAC's S3DF Kubernetes cluster behind single sign-on.
- 13REST + SSE endpoints
- 4 · 3LLM providers · vector stores
- 12 KBdrop-in widget, MIT
Why
Every group has the same onboarding problem: what a new student needs to know is spread over a decade of PDFs, and the people who could answer are busy. The obvious fix is a RAG chatbot, but the prototypes that get built serve only their author. A group needs plug-and-play: point it at a folder of documents, embed one script tag, and be able to tell whether the answers are any good. The project began as a Stanford ICME research-rotation prototype; my part was to turn it into that service.
How
The service is a FastAPI app whose /ask/stream endpoint streams status, retrieved documents and tokens over server-sent events. Knowledge bases are isolated per group and per embedding model on any of three vector backends: in-memory for development, OpenSearch with HNSW indices, or Qdrant. Generation is routed by a provider:model id across Ollama, OpenAI, Gemini and the Stanford AI API, with async streaming and automatic fallback.
The part I care most about is the evaluation loop. After each stream closes, the exchange is queued and written in batches to monthly-rotated OpenSearch indices, and a sidecar scores it with RAGAS faithfulness and answer relevancy. Nothing sits on the user's request path, and a log viewer filters answers by knowledge base, model and score, so a bad week is visible rather than anecdotal.
What it does
- In production. Four services on the S3DF Kubernetes cluster (about 15% of a 384-core cluster) behind SLAC SSO. Getting there meant fixing OOM-killed embedding jobs and ReadWriteOnce volume conflicts with single, vertically scaled replicas, then shipping thirteen date-tagged images in six weeks with one-command deploys.
- Embeddable. A script tag and a config object give any static page streaming Markdown and LaTeX answers, a collapsible sources panel, a RAG switch, dark mode and local history.
What I did. Top contributor to OuRAGboros (54% of commits): the REST and streaming API, multi-knowledge-base isolation, provider routing, the evaluation loop, and the Kubernetes deployment and SSO layer. Sole author of Chatbot-Widget. Embedding fine-tuning was a collaborator's work; I served and hot-swapped the resulting models.
Production instance: mli-llm-rag.slac.stanford.edu, behind SLAC single sign-on. Commit shares from the GitHub contributor graphs, August 2026.