Session
An AI Safety Accord: The Role of Investors
Wednesday, October 7, 2026 · 4:05 PM – 5:05 PM ET
Chapel
About this session
A June 2026 survey of AI experts, conducted by MIT, assigned a 20% chance that misaligned models would precipitate catastrophic harm (defined as more than 1 million human deaths, more than $100B in damage, or civilization-scale intangible harms) before 2030 unless preventive actions are taken. And yet, frontier AI developers are rolling back their safety commitments. The rollbacks center on “competitor contingency:” the position that a company need not honor safety commitments when its competitors’ commitments are weaker. Anthropic’s Responsible Scaling Policy 3.0 (February 2026) replaced its prior unconditional commitment to pause development at defined risk thresholds with “competitor-contingent” commitments. OpenAI’s updated Preparedness Framework permits adjusting safety requirements if a competitor releases a high-risk system. The Future of Life Institute’s Summer 2026 AI Safety Index concluded that these rollbacks have “undermined safety frameworks across the board” and incentivize “a collective race to the bottom” that could generate unacceptable systemic risk for investors.
Competitor contingency can be reined in through universal agreements that governments and companies sign onto. Like arms-control regimes, these agreements convert unilateral disadvantage into collective restraint. Momentum exists: global AI governance efforts now extend to the G7 and the UN; in a July 2026 Financial Times op-ed, OpenAI’s Sam Altman called for an international body modeled on the International Atomic Energy Agency; Anthropic’s Dario Amodei has proposed FAA-style mandatory testing with government authority to block unsafe deployments; and Nobel Prize winner Demis Hassabis, co-founder of Google Deep Mind, favors a hybrid public-private regulatory regime modeled on FINRA, the U.S. Financial Regulatory Authority. When rival CEOs independently call for external regulation, self-governance has visibly failed. But these efforts will fall short if companies resist due to competitive concerns. This is where investors must lead.
Investors have leverage to strengthen AI safety by exerting pressure on their investee companies to push for, and join, a universal agreement. Frontier development requires enormous capital for data centers, giving both equity and fixed income investors the ability to condition financing on safety governance. Many leading AI companies are private; the limited partners behind venture capital and private equity funds are diversified asset owners for whom systemic risk cannot be diversified away. Sovereign debt investors can press governments directly. Yet according to an April 2026 Oxford Martin AI Governance Initiative report based on roundtables with 62 institutional investors, “[i]nvestors are currently placing limited focus on the uncertain and potentially catastrophic impacts of powerful AI…. This contrasts with the AI governance and safety community which primarily focuses on these risks.”
With this session, we’ll offer:
• For investors new to AI safety, a plain-language explanation of frontier AI systemic risk and why fiduciary duty requires attention to it;
• For those already engaging tech companies, a survey of current frontier safety frameworks, the competitor-contingent rollbacks at Anthropic, OpenAI, Google DeepMind, and Meta, and how to hold companies to their own stated red lines;
• For all, engagement approaches across equity, fixed income, private markets, and sovereign debt — to press for corporate and government participation in a binding international AI safety regime.
Next session
Platform Worker Solidarity in the Age of AI