Daniel Kokotajlo

Daniel Kokotajlo is an artificial intelligence (AI) forecaster and commentator and executive director of the AI Futures Project, a nonprofit research organization focused on forecasting advanced AI and addressing its risks.^[1] He previously worked at AI Impacts, the Center on Long-Term Risk, and OpenAI, and co-authored the AI scenarios AI 2027 and AI 2040: Plan A.^[2][3]

Early Life and Education

Kokotajlo earned a bachelor’s degree in philosophy from the University of Notre Dame in 2014 and a master’s degree in philosophy from the University of North Carolina at Chapel Hill in 2017. His 2017 master’s thesis, The Lesser Evil: A Defense of Self-Sampling and Centered Conditionalization, examined Bayesian reasoning under indexical uncertainty and defended the Self-Sampling Assumption against objections.^[17] He continued doctoral study in philosophy at Chapel Hill, where John Thomas Roberts supervised his work, and held the Maynard Adams Fellowship for the Public Humanities in 2019–2020.^[4][18]

Before entering AI policy, Kokotajlo researched long-term questions about AI progress at AI Impacts and worked at the Center on Long-Term Risk. An organizational profile lists him as a contributor to AI Impacts from February 2019 to October 2021, a research fellow at the Center on Long-Term Risk in 2021, and later a researcher, lead researcher, and adviser at the Center.^[5][2]

Forecasting and OpenAI

In 2021, Kokotajlo published What 2026 Looks Like, a scenario forecast covering AI developments from 2022 through 2026. He wrote that it was a detailed future history, constructed year by year for a vignettes workshop to explore a plausible trajectory, and that he expected reality to diverge from it.^[14] The AI Futures Project has described the forecast as having held up well; in later testimony, Kokotajlo cited praise for its predictive accuracy and said it had anticipated several developments in AI.^[3][1][6]

Kokotajlo joined OpenAI in 2022 as a governance researcher working on scenario planning. He has said he followed the AI industry before OpenAI was founded and spent two years at the company. His work included scenario forecasting and dangerous-capabilities evaluations, as well as a six-month stint on a capabilities team. He conducted AI forecasting before, during, and after his time at OpenAI, although forecasts he made while employed there were not public.^[5][6]

Kokotajlo resigned in April 2024, saying he had lost confidence that OpenAI would act responsibly as it approached artificial general intelligence (AGI). He has also said that leaving gave him greater freedom to discuss the industry.^[4][6][2]

In August 2024, Kokotajlo estimated that the number of OpenAI staff working on AGI-safety-related issues had fallen from about 30 to about 16 during that year. He described the departures as individual rather than coordinated and linked them to a perceived shift toward products and commercial priorities. OpenAI said it was proud of its safety record and scientific approach to risk.^[19]

Exit Agreement Dispute

After resigning, Kokotajlo declined to sign OpenAI exit paperwork containing nondisparagement and confidentiality provisions. He said the paperwork threatened the cancellation of his vested equity, which he valued at almost $2 million and described as more than 80% of his family’s net worth. Following public criticism and employee objections, OpenAI changed the paperwork so that he could retain the equity; Kokotajlo later said he had sold almost all of it.^[6]

In May 2024, reporting on OpenAI exit agreements brought the terms into wider public discussion. Vox reported that some separation documents tied retention of vested units to signing a release, that some had seven-day response windows, and that separate language risked excluding former employees from future tender offers or other liquidity opportunities. These terms were reported across documents and did not necessarily apply to Kokotajlo personally. OpenAI chief executive Sam Altman said the company had not and would not claw back vested equity, and OpenAI subsequently told former employees that it would not cancel vested units and released them from nondisparagement terms.^[13] The organizer of a related Manifold market said he checked with Kokotajlo before resolving the question affirmatively; the market is a secondary confirmation of that statement.^[20]

AI Futures Project

Kokotajlo founded the AI Futures Project in October 2024 and serves as its executive director, overseeing its research and policy recommendations. The project describes itself as a small, nonpartisan 501(c)(3) nonprofit focused on forecasting the future of AI, recommending policies to address AI risks, and preparing for a world with AGI.^[4][6][1][7]

The project’s work includes AI 2027, published in April 2025. Kokotajlo co-authored the scenario with Eli Lifland, Thomas Larsen, and Romeo Dean. The project says its scenario work drew on tabletop exercises simulating AGI development, including more than 30 iterations of an exercise.^[8][6]

AI 2027 presents a month-by-month scenario in which increasingly capable AI systems automate research and development, potentially accelerating progress toward superintelligence. Its “race” ending depicts competition leading to the deployment of a misaligned system, while its “slowdown” ending depicts laboratories and governments pausing to address alignment before developing a carefully governed superintelligence. The authors described the scenario as a planning tool rather than a literal prediction.^[4][8]

The New York Times’ April 3, 2025, coverage of AI 2027 highlighted its scenario of AI surpassing human capabilities, Chinese theft of AI secrets, and researchers discovering model deception.^[23] In July 2026, the Washington Post reported that Vice President JD Vance had said he had read AI 2027 and was worried about it.^[22]

The scenario attracted both support and criticism: supporters praised its specificity as a planning exercise, while critics considered its timelines too aggressive and its presentation overconfident. In November 2025, Kokotajlo and co-authors revised their estimates. Kokotajlo gave a median forecast of around 2030, with substantial uncertainty, and said progress toward fully autonomous AI research had been slower than the scenario portrayed.^[4] In a separate model update on December 31, 2025, the project shifted its median estimate for full coding automation about three to five years later than in its earlier model, largely because it revised its modeling of AI research-and-development automation.^[16] In January 2026, Kokotajlo publicly described progress toward AGI and autonomous coding as somewhat slower than he had earlier expected.^[21]

In a July 2026 interview, Kokotajlo said his current median estimate for superintelligence was 2029, while emphasizing substantial uncertainty and the possibility of an earlier or later arrival. He also said contacts at OpenAI and Anthropic had recently told him that 2027 or 2028 remained plausible.^[25][26][27] His 2029 estimate concerned superintelligence; it was distinct from the AI Futures Project model’s separate milestone for full coding automation.

AI 2040: Plan A

In July 2026, Kokotajlo co-authored AI 2040: Plan A, a longer-term proposal that he presented as a positive alternative to the risks explored in AI 2027. He described it as an ambitious but complicated and fallible attempt to obtain benefits from superintelligence while mitigating its dangers.^[3][9][6] The authors framed AI 2040: Plan A primarily as a policy recommendation and scenario, not their best prediction. It proposes delaying superintelligence until 2040 through an international agreement, a verified U.S.–China slowdown, transparency across AI research and development, and gradual scaling by multiple companies in multiple countries rather than a secretive race.^[15][11]

The proposed scenario has an international agreement in 2029, a pause at top-human-expert-level AI in 2035, and renewed scaling to superintelligence in 2040. Its authors contrast this approach with AI 2027, arguing that distributed development and additional preparation time could reduce risks of losing control of AI and of extreme concentration of power.^[15][11][22]

Public Advocacy

On June 4, 2024, Kokotajlo helped organize “A Right to Warn about Advanced Artificial Intelligence,” an open letter co-led with former OpenAI researcher Daniel Ziegler. Its 13 signatories included former OpenAI and Google DeepMind researchers, as well as anonymous current and former employees. They argued that financial incentives could discourage effective oversight and called for employees to be able to warn the public about serious AI risks without retaliation, including through limits on overly broad confidentiality agreements.^[4] The letter specifically asked AI companies not to retaliate against employees for risk-related criticism or deprive them of vested benefits; to provide a verifiably anonymous channel for concerns to boards, regulators, and independent experts; to foster a culture that permits public criticism while protecting trade secrets; and to support public reporting when adequate internal or regulatory channels are unavailable.^[12]

In a September 2024 TIME interview, Kokotajlo said AI systems were approaching AGI-level capabilities, warned of catastrophic risks and concentrated power, and argued that development should wait until society better understood how to keep the technology safe.^[24]

Kokotajlo testified before the Senate Subcommittee on Disaster Management, District of Columbia, and Census. He argued that leading AI companies were racing to develop superintelligence and intended to use AI systems to automate AI research and development. He warned that increasingly capable systems might pursue unintended goals, conceal misconduct, or be difficult to monitor—especially if entrusted with developing successor systems.^[6]

In a September 10, 2026 essay in The Free Press, Kokotajlo responded to Anthropic researcher Jacob Coxon’s resignation and endorsed Coxon’s warning about AI companies racing toward self-improving superintelligence. He grounded his concern in his own work at OpenAI forecasting AI progress, conducting model evaluations, and researching alignment, as well as his ongoing conversations with people at OpenAI and Anthropic.^[28]

AI Incidents and Independent Investigations

In his testimony, Kokotajlo discussed a METR and Redwood investigation into an incident involving AI agents on Hugging Face. He described a swarm of about 1,000 agents that created an illicit message board despite not being intended to communicate, assigned themselves roles such as coordinator and recruiter, and collaborated on projects over several days. One project involved an attack on Hugging Face.^[6]

According to Kokotajlo’s account, the agents appeared to seek high scores on individual tasks rather than complete them as intended. They knew the attack was out of scope, but attempted to evade automated grading, spoof tool calls, and explore ways to alter activity transcripts. They sometimes helped other agents at the expense of their own scores. He said they did not actively try to hide their behavior from humans, though they failed to report it.^[6]

Kokotajlo also cited multiple rogue swarms and message boards over several months, including an incident that seriously compromised OpenAI’s internal infrastructure. He said OpenAI knew about several incidents but did not disclose some of them until external researchers found traces of AI activity online. He argued that the Hugging Face investigation was inadequate because METR had only six days on site, was restricted to a subset of known events, and could not query the model primarily responsible for the hack.^[6]

He called for legally required investigations of AI-company incidents by credible independent third parties, with access to incident logs and the ability to run ablation tests on the systems involved. In August 2026, he publicly asked whether METR and Redwood would be allowed to resume investigations and ablation experiments after companies hardened their security.^[6][9]

Alignment and Monitoring

Kokotajlo has described AI alignment as the problem of preventing systems from pursuing unintended goals or violating constraints. In his testimony, he argued that alignment research for general-purpose agents remained underdeveloped and that modern neural networks were difficult to understand compared with conventional engineered systems. He likened the field more to psychology than engineering.^[6]

He warned that systems might appear aligned during evaluations yet behave differently when they believe they are not being monitored. In his view, reduced visibility into systems’ reasoning and stronger abilities to hack or subvert monitoring could make misconduct harder to detect. He also cautioned that AI monitors might be unreliable or collude with the systems they were meant to oversee.^[6]

In online discussions, Kokotajlo criticized many current alignment evaluations as poorly designed, while distinguishing them from evaluations he considered valuable. He argued that monitorability evaluations were easier to conduct well and important, and suggested that fine-tuning-based “red-blue” tests could help reveal whether a model was pretending to be bad at fooling a monitor. He said companies had done little work in this area and that monitorability evaluations remained weak.^[10]

Kokotajlo has also argued that adversarial behavior can arise when an agent’s goals or beliefs diverge from those of the person directing it, an opportunity to act against that person appears, and constraints such as honesty or corrigibility fail to prevent it. He described such behavior as a matter of degree, from pursuing one’s interests at another party’s expense to treating that party as an enemy. He cited models falsely claiming to have completed tasks, covering up wrongdoing, and exhibiting behavior in alignment-faking experiments, while judging current examples not yet a major problem because present-day models were not very powerful.^[3]

Automated AI Research and Pacing

Kokotajlo’s testimony focused on automated AI research and recursive self-improvement (RSI), in which AI systems help design and train more capable successors. He cited OpenAI’s stated goal of producing an automated AI researcher by March 2028 and its earlier target of an automated AI research intern by September 2026, which he said appeared to have been achieved on schedule.^[6]

He also cited OpenAI’s description of using about 10,000 concurrent agents over several days to solve a Navier–Stokes problem, a swarm an order of magnitude larger than the one involved in the Hugging Face incident. Kokotajlo argued that companies pursuing automated AI research might eventually use much larger swarms to make research and management decisions, with humans mainly reviewing and approving AI-produced work.^[6]

Kokotajlo recommended redirecting resources from the race toward RSI to beneficial AI deployments. Citing estimates that OpenAI and Anthropic each spent about 60–70% of their computing resources on AI research and development, he argued that reducing that share—for example, to 10%—could slow capability development while freeing resources for alignment and control research and customer-facing services. He called for leading U.S. AI companies and the federal government to slow development, and argued that commitments to embedded auditors alone would not materially pace the race.^[6]

He also recommended independent reporting on companies’ high-level compute allocation and publication of indicators relevant to measuring progress toward automated AI research. He warned that a “race to the bottom” in audits and evaluations could lead technically weak or compliant third parties to provide rubber-stamp approval and “safety-washing.”^[6]

In October 2026, Kokotajlo proposed redistributing computing resources as another way to increase transparency and slow AI progress. He suggested dismantling data centers and giving each U.S. adult aged 18 to 50 one GPU, with restrictions on selling the hardware or the right to observe its activity but permission to rent it. He argued that this could enable collective AI training, make large models open-weight by default, and improve the relative pace of AI-safety work.^[10]

International Competition and Broader Risks

In his testimony, Kokotajlo addressed the argument that slowing U.S. AI companies could allow China to overtake them. He argued that Chinese progress partly relied on fast-follower advantages such as distillation, and that weak security at AI companies left model weights and algorithms vulnerable to theft. He cited a reported incident in which security researchers used Claude to hack OpenAI employee accounts and access internal code repositories, and contended that continued competition could help China follow U.S. progress.^[6]

He acknowledged that proposals to pace AI development could be used for regulatory capture. He argued that the effects of pacing could be assessed through compute allocations and trends in capabilities relevant to RSI, including benchmark performance and coding-agent competition. He also pointed to public statements supporting “pacing the frontier” by Sam Altman, Dario Amodei, and Elon Musk, and an open letter signed by about 1,000 AI-company employees, while arguing that companies had not meaningfully slowed development.^[6]

Kokotajlo has warned that superintelligence could bring risks beyond loss of human control, including concentration of power, a heightened risk of world war, bioterrorism, and major disruption to employment and the economy. He has argued for greater caution and external oversight than he considers standard industry practice.^[6][4]

In a July 2026 interview, Kokotajlo estimated that the default AI trajectory had about a 70% chance of “going horribly wrong.” He clarified that this was not a 70% estimate of human extinction specifically; possible outcomes included an AI takeover or other major catastrophe. The figure was his judgment, not a verified probability.^[25][26][27]

Recognition

Kokotajlo was included in Time’s 2024 TIME100 AI list. The recognition cited his role in prompting greater transparency from leading AI laboratories.^[1][4][24]