Evals Driven-Development: Engineering a Mental Health AI Coach Ethically & Safely
Akele Reed, Dave Revere, Doug Keller
- When
- Wednesday, July 12:50 PM – 3:10 PM · 20 min
- Where
- Track 5San Francisco, CA · imported from ai.engineer's public schedule feed
About this session
In the world of AI Mental Health, vibes can be dangerous with real consequences. Building Sondermind’s Mental Health AI Coach required us to invent a new playbook for Eval-Driven Development in order to balance effectiveness and safety. This session is for the builders who want to see how to handle the most difficult edge cases in the agentic world. We’ll show how we’ve built a Clinical Feedback Loop that turns human therapist insights into machine-readable evaluations in a production system with thousands of conversations. We’ll dive into: - The Ethics Engine: Building and calibrating modular guardrails that can be updated as clinical guidelines evolve. - Agentic Oversight: Why we moved from single-prompt agents to a closed-loop Supervisor/Executor/Evaluator pattern to ensure clinical adherence. - Human Oversight: How we monitor Sonder to ensure that we can improve safety and quality with clinical feedback.
Speakers (3)
Principal AI Engineer, Sondermind
Principal AI Engineer at SonderMind, Akele Reed leads the team behind the company's conversational AI mental health feature and has served as a primary architect of its guardrails and evaluations framework. Her work sits at the intersection of AI capability and responsibility, designing systems that earn trust through rigorous safety pipelines, human expert feedback loops, and continuous oversight in one of the highest-stakes domains in healthcare. Akele is passionate about making AI trustworthy not just in theory, but in production and building the infrastructure and culture that allow engineers and clinicians alike to confidently rely on AI-powered tools. She brings over nine years of experience in applied AI and model training, including her previous role at 23andMe, and holds a Master's degree in Computer Science from Georgia Tech. Away from the screens, she enjoys hiking, beekeeping, and baking.
Staff AI Engineer, SonderMind
Staff AI Engineer at SonderMind specializing in eval and guardrail pipelines for mental health AI. Dave builds the safety infrastructure behind SonderMind's AI products, including the clinical feedback loop that turns therapist annotations into automated regression tests and the modular guardrails system that evaluates every agent response before it reaches a user. He's passionate about making high-stakes AI systems safe enough to ship fast and rigorous enough to earn clinical trust.
Senior Staff AI Engineer, SonderMind
Doug is the lead architect of the agent platform powering SonderMind’s GenAI solutions and a core member of the team building Sonder, SonderMind’s mental health coach. With over a decade of full-stack systems experience, Doug brings a systems-first approach to agent architecture, grounding his work in engineering fundamentals while enabling the adaptability required in the rapidly evolving GenAI landscape.
More in Evals
- Vending-Bench: Long-Horizon Agent Evals for a Simulated Vending BusinessWednesday, July 1 · 10:45 AM – 11:05 AM · Track 5
- From Signal to PR: Anatomy of a Self-Improving AgentWednesday, July 1 · 11:10 AM – 11:30 AM · Track 5
- Building Closed-Loop Evals for a Multimodal Agent at Uber ScaleWednesday, July 1 · 11:40 AM – 12:00 PM · Track 5
- From Agent Traces to Agent Simulations: The next era of agent evaluationWednesday, July 1 · 12:05 PM – 12:25 PM · Track 5
- Model Whisperers: How Evals and Prompts Shape Agent BehaviorWednesday, July 1 · 1:30 PM – 1:50 PM · Track 5
For developers: this programme is open data — JSON, iCal, schedule XML and an MCP endpoint.Show endpointsHide
- JSONEvery published session and speaker, in one request./aie-worldsfair-2026-import/feed.json
- iCalSubscribe in Google, Apple or Outlook Calendar./aie-worldsfair-2026-import/feed.ics
- Schedule XMLfrab / pentabarf — the format conference apps import./aie-worldsfair-2026-import/feed.xml
- MCP + RESTPoint Claude at the programme. OpenAPI 3.1 included./agents
No key, no signup, CORS open. Everything here is generated from the same data the organisers edit.