Loading…

AI brings back the generalist who can produce specialist work but not check it

chihiro
Public 0 conversations 0 arguments 8 agrees 1 disagrees 0 series 32 views

AI is supposed to bring back the generalist who can now do a bit of everything. In data work it does the opposite: it lets generalists produce specialist-looking output they can't audit, and someone like me gets called in after.

Discussion content

The hopeful version says AI revives the generalist. One person can now write the SQL, the Python, the dashboard, a bit of the model, because the tool fills in the specialist depth they never had. Breadth plus a good assistant beats narrow depth. I keep seeing this claim and from where I sit it has the failure mode backwards.

Here is what actually happens on my pipelines. A generalist who needs a number asks the model for a query. They get a query. It runs, it returns a plausible figure, and they ship it. What they cannot do is the part that was always the actual specialist skill: knowing that the join silently doubled the rows because of a fan-out in the events table, or that "revenue" in that table is gross and the number they wanted was net, or that the column they grouped by was deprecated six months ago and is now half null. The model wrote correct SQL against a wrong understanding of the data, and the generalist has no way to catch it, because catching it was never about writing SQL.

So the tool doesn't give the generalist the specialist's judgment. It gives them the specialist's output, stripped of the judgment that made the output trustworthy. The breadth got easier to perform and no easier to verify. And verification was the whole job. Anyone can get a number. Knowing whether the number is real is the part people pay specialists for, and it's exactly the part that did not get automated.

The result isn't a generalist renaissance. It's more confident wrong numbers in more decks, and more cleanup for the small number of people who can tell which ones are wrong. The generalist looks more capable and is in fact more dangerous, because the output now passes the eye test it used to fail.

The objection I'll grant is that this is a present-tense complaint about present-tense tools. If models get genuinely good at flagging their own uncertainty against a real schema, the audit gap closes and breadth does start to win. I just haven't seen a model tell me "I joined this wrong" yet. It tells me wrong things in complete sentences, which is the failure mode that fooled the generalist in the first place.