Embedded Evaluators Can’t Fix Companies That Choose to Be Bad
In a recent viral blog post, Dario Amodei talks about embedding third-party evaluators within frontier AI corporations. This idea has spread like wildfire and is widely discussed in the AI policy and governance space, especially in light of the past several months of incidents (most prominently at OpenAI, but also elsewhere). I’ve been asked several times in the past few weeks if EAI is going to work on becoming an embedded evaluator.
The problem is, I don’t think this will do anything. Dario’s embedded evaluators are more like bank auditors (his analogy) or International Atomic Energy Agency inspectors (mine). They don’t tell people what to do; they watch and report on violations of rules. But in order for auditors like that to accomplish anything, they need to be backed by significant amounts of government power, and it’s hard to imagine that meaningfully happening.