RAG · Multi-agent systems · LLM evaluation
Four years building production Generative AI — RAG chatbots, multi-agent workflows and LLM evaluation pipelines. Currently building AI products for LMS platforms at Red Nucleus; before that, an eight-agent QA framework that automated up to 80% of manual validation, and small language models fine-tuned onto customers’ own secured hardware.
Experience
Production systems, private repositories. What I owned and what changed.
AI Engineer
Production AI products for Learning Management System platforms — retrieval, evaluation and agentic personas, shipped to multiple clients.
Senior Consultant — AI Engineering
Two Gen AI platforms for enterprise quality engineering, shipped into customer environments — including air-gapped ones where a hosted model API was never an option.
Selected work
Written up from the code: the architecture, the trade-offs, and what broke.
A personal AI-news triage agent that runs itself every morning on GitHub Actions — clusters sixteen sources, scores what survives against a written profile of one reader, and refuses to fill the digest when nothing is worth reading.
Trajectory evaluation for AI agents — grade the path the agent took, not just the answer it returned. Including the one path nobody grades: a privileged tool call whose arguments came from a web page the agent read.
A document chatbot that reads the pictures too — flowcharts, tables and architecture diagrams are described by a vision model, indexed as text, and retrieved alongside the prose. Every answer is scored before it reaches the screen.
Ask a PostgreSQL database a question in English and get an answer — a ten-node LangGraph pipeline that plans, writes, checks and repairs its own SQL before anything touches the database.
An AI course-generation platform: upload a document or name a topic, and a five-agent pipeline produces a full course in any of eight formats, with a quiz and a graded assessment.
A fitness platform where two LangGraph agent teams write personalised training and nutrition plans — eleven specialists, each owning one domain, converging on a single plan.
Approach
A retrieval change that improves vibes and degrades recall is a regression. I build the harness before I tune the system, so improvement is a number rather than an impression.
A model choice is a three-way trade between quality, p95 latency and cost per call. I price that trade explicitly instead of defaulting to the largest model available.
Language models fail confidently. The interesting engineering is in detecting low-confidence output, falling back gracefully, and making the failure legible to the user.
Contact
If you're building something where model reliability actually matters, I'd like to hear about it — including the parts that aren't working yet.