Your Agent Evolved. Your Evals Didn't.
Ameya Bhatawdekar
- When
- Tuesday, June 3011:10 AM – 11:30 AM · 20 min
- Where
- Leadership 2San Francisco, CA · imported from ai.engineer's public schedule feed
About this session
Knowing which generation your agent is in, which failure modes your current evals are blind to, and what to build next is the difference between shipping with confidence and flying blind. Agent architectures have evolved through six generations; prompt, chain, ReAct loop, workflow graph, modern agent loop, AI harness. And each one quietly breaks the eval strategy of the generation before it. A prompt-quality rubric won't catch a bad tool call; a trace scorer won't catch memory poisoning. Using a single SRE incident response agent threaded through every generation, this talk shows exactly where each architecture outgrows its evals and what you need to close the gap.
Speaker
VP, Field CTO, Braintrust
Ameya Bhatawdekar is VP, Field CTO at Braintrust, where he helps teams evaluate and observe production AI systems. He previously led machine learning work at Dropbox and focuses on making AI-powered features reliable through evals and observability.
More in AI Architects: Show my Workflow
- The Last Human Code Review: Building Trust in AI-Generated CodeTuesday, June 30 · 11:40 AM – 12:00 PM · Leadership 2
- Prototyping as Leadership: How a CTO Ships with AI AgentsTuesday, June 30 · 12:05 PM – 12:25 PM · Leadership 2
- Serving 2 Million Models Without Melting: Scaling the Hugging Face HubTuesday, June 30 · 1:30 PM – 1:50 PM · Leadership 2
- IT Admin for the AI Workforce: Why Your AI Agents Will Need Their Own IT DepartmentTuesday, June 30 · 1:55 PM – 2:15 PM · Leadership 2
- The Era of Compound EngineeringTuesday, June 30 · 2:25 PM – 2:45 PM · Leadership 2
For developers: this programme is open data — JSON, iCal, schedule XML and an MCP endpoint.Show endpointsHide
- JSONEvery published session and speaker, in one request./aie-worldsfair-2026-import/feed.json
- iCalSubscribe in Google, Apple or Outlook Calendar./aie-worldsfair-2026-import/feed.ics
- Schedule XMLfrab / pentabarf — the format conference apps import./aie-worldsfair-2026-import/feed.xml
- MCP + RESTPoint Claude at the programme. OpenAPI 3.1 included./agents
No key, no signup, CORS open. Everything here is generated from the same data the organisers edit.