NEW EBOOK |
 |
The weakest AI
model won. It had a semantic layer. |
What seven AI models scored on BIRD-Interact, with and without AtScale. |
|
Download the Ebook |
|
Ask an AI to rank the stars in the galaxy. It sorts them A to Z. Tell it that’s wrong, and it sorts them by size. You meant brightness, and it never asked. |
BIRD-Interact is a public benchmark built on questions like that: 600 vague business tasks on 22 messy databases. The scoring is strict, so a close answer scores zero. |
We ran seven AI models through it, once with our semantic layer and once without. Every model improved. With AtScale, the weakest model beat the strongest one on its own, and Claude Sonnet 5 doubled its accuracy on the business questions that have a right answer.* |
Dave Mariani, Jay Schuren, and Mark Palmer wrote up every score in a new ebook, along with the questions we still got wrong and how to fix each one. |
Inside the eBook |
 |
How seven models scored with and without AtScale |
|
|
Why the cost of a right answer fell for all seven |
|
|
How 21 lines of AI-generated SQL turn into 3 |
|
|
The five kinds of questions we still missed, and the fix for each |
|
|
|
Download the Ebook |
*Preliminary results from AtScale’s October 2026 runs on the public BIRD-Interact harness; not yet verified by the BIRD team. Full detail at atscale.com/bird. |
|
|
