RAG chatbot for research groups

Plug-and-play research assistant, evaluated on every answer.

A research group's papers, notes and internal documents, turned into a chatbot that answers with citations and drops into the group's own web page with a script tag. OuRAGboros is the service; Chatbot-Widget is the 12 KB front end. Both are open source under SLAC-ML, and the service runs on SLAC's S3DF Kubernetes cluster behind single sign-on.

Why

Every group has the same onboarding problem: what a new student needs to know is spread over a decade of PDFs, and the people who could answer are busy. The obvious fix is a RAG chatbot, but the prototypes that get built serve only their author. A group needs plug-and-play: point it at a folder of documents, embed one script tag, and be able to tell whether the answers are any good. The project began as a Stanford ICME research-rotation prototype; my part was to turn it into that service.

How

Architecture: the widget talks to a FastAPI streaming service, which retrieves from one of three vector stores and generates through one of four LLM providers; an asynchronous logger and RAGAS evaluator score every answer; all of it runs on S3DF Kubernetes behind SSO SLAC S3DF Kubernetes · four services · behind SLAC single sign-on Chatbot-WidgetFastAPIRetrieverLLM router Query loggerOpenSearch log indicesRAGAS evaluator 12 KB, any static pageMarkdown · LaTeX · sources panel REST + server-sent events13 endpoints isolated per knowledge base × modelin-memory · OpenSearch · Qdrant provider:model ids · async · fallbackOllama · OpenAI · Gemini · Stanford AI API async queue, batched writesrotated monthlyfaithfulness · answer relevancy SSEquerysources + promptanswer tokens stream back after the stream closesscores
Request path across the top; the evaluation loop, in accent, never sits on it.

The service is a FastAPI app whose /ask/stream endpoint streams status, retrieved documents and tokens over server-sent events. Knowledge bases are isolated per group and per embedding model on any of three vector backends: in-memory for development, OpenSearch with HNSW indices, or Qdrant. Generation is routed by a provider:model id across Ollama, OpenAI, Gemini and the Stanford AI API, with async streaming and automatic fallback.

The part I care most about is the evaluation loop. After each stream closes, the exchange is queued and written in batches to monthly-rotated OpenSearch indices, and a sidecar scores it with RAGAS faithfulness and answer relevancy. Nothing sits on the user's request path, and a log viewer filters answers by knowledge base, model and score, so a bad week is visible rather than anecdotal.

What it does

What I did. Top contributor to OuRAGboros (54% of commits): the REST and streaming API, multi-knowledge-base isolation, provider routing, the evaluation loop, and the Kubernetes deployment and SSO layer. Sole author of Chatbot-Widget. Embedding fine-tuning was a collaborator's work; I served and hot-swapped the resulting models.

Production instance: mli-llm-rag.slac.stanford.edu, behind SLAC single sign-on. Commit shares from the GitHub contributor graphs, August 2026.