Building a science of open, understandable, and safer AI
My research asks how large models acquire their capabilities, how we can shape that process, and who gets to participate in answering those questions. I work across three closely connected research programs.
Open-Weight Safety
Most AI safety research assumes a closed model served through an API. In that setting, a provider can monitor users, filter inputs and outputs, revoke access, and push updates. Those assumptions do not hold when model weights are released and downstream users can inspect, modify, and deploy them independently.
I want to help establish open-weight safety as its own field: safety research that is technologically compatible with unrestricted access to model weights and philosophically compatible with the open-source principle that anyone may use a system for any purpose. This means developing safeguards that remain meaningful after release rather than relying on surveillance, use restrictions, or refusal behavior that can be removed through fine-tuning.
Deep Ignorance demonstrates one promising approach: filtering pretraining data to prevent a model from acquiring a narrowly targeted capability in the first place. Filtering is not a universal solution. Its usefulness depends on whether the capability can be separated from valuable knowledge, whether it is difficult to reconstruct from benign data, and whether safe behavior requires understanding it. I am also interested in durable structural interventions, positive demonstrations, controllability for downstream deployers, and rigorous methods for machine unlearning.
Interpretability Over Time
Most interpretability research studies a finished model: given an input and an output, it asks what computation connected them. I am more interested in the causal question underneath that computation: why did the model learn to behave this way in the first place?
I study models as snapshots of a learning process shaped by data, architecture, objectives, and optimization. A mature science of AI should let us predict important properties from early training signals, intervene when a trajectory is going wrong, and ultimately design training procedures that reliably produce the properties we want. This requires longitudinal experiments, controlled comparisons, and access to the intermediate states that conventional model releases discard.
Pythia was built as infrastructure for this kind of science: a controlled suite of models trained on identical public data, with 154 checkpoints released for every model. Don’t Just Fix It in Post develops the broader argument that the proper object of scientific study in machine learning is not only the trained model, but the process that produced it.
PythiaDon’t Just Fix It in PostEmergent and Predictable Memorization in Large Language ModelsLLM Circuit Analyses Are Consistent Across Training and Scale
Open Science Beyond the Largest Companies
Scientific understanding cannot depend on the permission of the companies building the systems under study. When the most capable models, training data, and computational resources are controlled by a handful of corporations, researchers outside those institutions are pushed toward auditing finished products rather than investigating how those systems were made.
EleutherAI began as a response to that concentration of power. We trained and released large models because independent researchers needed meaningful access to the technology—not merely an API through which a corporation could determine which questions were possible to ask. Projects including The Pile, GPT-NeoX-20B, Pythia, and OpenFold were designed to make important parts of modern AI inspectable, reproducible, and available for others to build upon.
For me, open science is both a scientific commitment and a political project. Public data, code, models, and training artifacts enable independent replication and criticism; they also allow academics, nonprofit researchers, auditors, and small organizations to participate in setting the research agenda. The goal is not openness for its own sake, but a research ecosystem in which knowledge about powerful technologies is not subordinate to the interests of trillion-dollar companies.
The PileGPT-NeoX-20BEleutherAI: Going Beyond “Open Science” to “Science in the Open”