Text to SQL went from a research toy that could barely fill a template to a feature you use without thinking.
The path from seq2sql to today's models explains why it finally works, and where it still doesn't.
The early text to SQL models
seq2sql and its cohort, around 2017, learned to map a question to a query over a single table, trained on datasets like WikiSQL.
Impressive for the moment, brittle in practice: one table, narrow schemas, and no joins worth the name. They mostly slotted values into a fixed query shape rather than reasoning about the database.
Ask one for a three-table join and it simply couldn't express it. The query space it searched didn't include the answer.
The benchmark era
Spider raised the bar with multi-table, cross-domain schemas the model had never seen in training. Execution accuracy, does the query return the right rows, became the number people reported instead of string-matching the SQL.
Specialized models improved fast, but real, messy databases with cryptic column names still tripped them up.
The LLM era
General models plus schema context now top harder benchmarks like BIRD, which deliberately adds dirty data, big schemas, and a gap between a query that runs and one that's right.
The model isn't SQL-specific anymore. The schema you feed it and the prompt around it do most of the work, which is why the same model can be brilliant on one database and hopeless on another.
Give it a clean, well-named schema and it soars. Give it a legacy mess and it stumbles, no matter how high it scored on a benchmark.
| Era | Representative | Handled | Main limit |
|---|---|---|---|
| 2017 | seq2sql / WikiSQL | Single-table questions | No real joins |
| 2018-2021 | Spider-era models | Multi-table, unseen schemas | Messy real data |
| 2023+ | LLMs + schema context | Cross-domain, harder queries | Still needs a clean, described schema |
Why the benchmarks kept getting harder
Each benchmark existed because the last one got solved. WikiSQL fell, so Spider added joins and unseen schemas. Spider scores climbed, so BIRD added dirty data, huge schemas, and even query efficiency, not just correctness.
The moving target is the whole story. Every time the models caught up, the field moved the goalposts to whatever real databases still broke.
What the trend means for you
Every jump came from feeding the system more real-world structure, not from a cleverer decoder. So the highest-leverage thing you can do isn't picking a model, it's documenting your schema and building a modeled layer.
It also means benchmark scores don't transfer to your warehouse. A model that tops BIRD can still fail on your tables if the columns are named c1 through c40 and nothing is documented.
So the practical move is boring: name your columns, document your metrics, and the model you already have gets noticeably better for free.
The line from seq2sql to today is a story about the schema, not the model. Each jump worked by feeding the system more structure, which is also why text to SQL still stumbles the moment your schema is a mess.
Want a stronger data analyst role or a raise? Grab the FREE Product Analyst Playbook and get the exact roadmap to your next offer.
