Benchmarks: The Good, the Bad, and the Ugly
Ali Khial
- When
- Wednesday, July 13:20 PM – 3:40 PM · 20 min
- Where
- Track 9San Francisco, CA · imported from ai.engineer's public schedule feed
About this session
We’ll explore the good, the bad, and the ugly of AI benchmarks: where they provide useful signal, where they create false confidence, and where data quality issues like contamination, label noise, narrow task design, and leaderboard gaming can mislead teams. The goal is not to dismiss benchmarks, but to use them better: as one part of a disciplined evaluation practice that connects model performance to real-world reliability.
Speaker
Head of AI/ML, G2i
Ali Khial is an engineering leader focused on building AI-native systems that work beyond the demo stage. He currently leads AI/ML at G2i, where he works across frontier AI evaluation, software engineering benchmarks, agentic workflows, and human-data quality systems. His current work centers on the gap between impressive AI prototypes and reliable production systems. He is especially interested in AI evaluation, data quality, tool-using applications, and the engineering practices needed to ship model-powered products in real-world environments.
More in Posttraining & Midtraining
- Building self-learning loops for your agentMonday, June 29 · 11:05 AM – 12:05 PM · Track 1
- Bugcrowd posttraining talkTuesday, June 30 · 12:05 PM – 12:25 PM · Track 9
- What's next after RLHF?Wednesday, July 1 · 10:45 AM – 11:05 AM · Track 9
- State of DataWednesday, July 1 · 11:10 AM – 11:30 AM · Track 9
- Learning on the job: the future of post-trainingWednesday, July 1 · 12:05 PM – 12:25 PM · Track 9
For developers: this programme is open data — JSON, iCal, schedule XML and an MCP endpoint.Show endpointsHide
- JSONEvery published session and speaker, in one request./aie-worldsfair-2026-import/feed.json
- iCalSubscribe in Google, Apple or Outlook Calendar./aie-worldsfair-2026-import/feed.ics
- Schedule XMLfrab / pentabarf — the format conference apps import./aie-worldsfair-2026-import/feed.xml
- MCP + RESTPoint Claude at the programme. OpenAPI 3.1 included./agents
No key, no signup, CORS open. Everything here is generated from the same data the organisers edit.