POLISCI 158 · AI & Politics · Session 16b · Monday, November 16, 2026
“We are not there yet, and recursive self-improvement is not inevitable.” Anthropic, “When AI builds itself”
Not there yet. How close, and who gets to decide?
Anthropic, “When AI builds itself,” Favaro and Clark, updated Sep 18, 2026
Why would they confirm RSI?
Why would they deny RSI?
A loop, a multiplier, and a story with two endings.
RSI: an AI system “capable of fully autonomously designing and developing its own successor”
Anthropic: “We are not there yet” · today humans still direct the loop
Anthropic, “When AI builds itself,” Favaro and Clark, updated Sep 18, 2026
“The AI R&D progress multiplier is how much faster AI software improvements are advancing with AI usage than without it.”
1x no AI help
2x twice as fast
10x “about a year of algorithmic progress every month”
The 10x example is from the scenario (Jun 2027), not a measurement
Kokotajlo, Alexander, Larsen, Lifland and Dean, AI 2027 (ai-2027.com), Apr 3, 2025
| Milestone | Date | Multiplier |
|---|---|---|
| Agent-1, “a scatterbrained employee who thrives under careful management” | Early 2026 | 1.5x |
| China nationalizes its AI research, “about six months behind” | Mid 2026 | 1.7x |
| Agent-1-mini, “10x cheaper” · “AI has started to take jobs” | Late 2026 | 2x |
| Superhuman coder (SC) | Mar 2027 | 5x |
| Superhuman AI researcher (SAR) | Aug 2027 | 25x |
| Superintelligent AI researcher (SIAR) | Nov 2027 | 250x |
| Artificial superintelligence (ASI) | Dec 2027 | 2,000x |
2026 multipliers from the scenario’s dashboard (Apr, Aug, Dec 2026) · 2027 dates from the racing ending · authors: “AI 2027 is not a recommendation or exhortation. Our goal is predictive accuracy.”
Kokotajlo, Alexander, Larsen, Lifland and Dean, AI 2027 (ai-2027.com), Apr 3, 2025
Fictional labs: OpenBrain and DeepCent
A scenario, written in April 2025 · dates are the authors’ forecast
Kokotajlo, Alexander, Larsen, Lifland and Dean, AI 2027 (ai-2027.com), Apr 3, 2025
Oversight Committee: “a joint management committee of company and government representatives”
| Race | Slowdown |
|---|---|
| “The committee votes 6–4 to continue internal use of Agent-4.” | Committee “votes 6–4 to slow down and reassess.” |
| “The problem was real and the fixes didn’t work.” | “This is enough evidence to finally shut down Agent-4.” |
| Humans lose control to misaligned AI | A small committee controls superintelligent AI |
Should a committee of executives and officials make this call?
“The slowdown ending is not a recommendation.”
Kokotajlo, Alexander, Larsen, Lifland and Dean, AI 2027 (ai-2027.com), Oct–Nov 2027 sections
Three labs, three kinds of evidence, and who gets to check it.
METR, Nov 5, 2026
11 weeks of high-access evaluation inside Anthropic, Aug 12–Oct 29, 2026
Capability index (AECI): +15 pts/yr → +29 pts/yr after the Oct 2025 break
“accelerated 1.9x [95% confidence interval: 1.5–2.8x] since October 2025”
“on well-specified tasks” only · index mixes internal and external benchmarks · sensitive to assumptions about benchmark difficulty
METR, Cunningham, Rush, Whitfill & Wijk, Nov 5, 2026
Share due to AI: “perhaps ≈80%” (METR’s guess, Oct 2026)
Range: “we would not be surprised if the true acceleration was closer to 1.3x or 3x the historical rate.”
“Anthropic redacted a majority of the evidence we found most informative for these conclusions.”
“the public has to rely on METR’s judgment much more than would be ideal”
Also: “Anthropic canceled a compute allocation component of this assessment.”
METR, Cunningham, Rush, Whitfill & Wijk, Nov 5, 2026
AI Futures Project, Nov 10, 2026
AI 2027 forecast: “2x AI R&D uplift in Dec 2026” = the engine
METR, as AI Futures reads it: “1.7x AI-attributable acceleration in the pace of progress” = the speed
“by this metric” · AI 2027 trajectory reconstructed by the authors · rests on METR’s estimate (1.3x–3x plausible, evidence partly redacted)
AI Futures Project, “Q3.5 Timelines Update,” Lifland, Halstead, Kokotajlo, Dean et al., Nov 10, 2026
Scale: AL0 no AI → AL3 collaborates → AL4 leads → AL5 fully automated (not reached)
Claude leads 26% of R&D work (Aug 2026), up from under 1% (Feb 2026)
“Claude is not operating fully autonomously for any measured subset of AI R&D work.”
Anthropic’s own measurement of its own work · not independently verified
Anthropic, “Measurements for understanding the pace of AI development inside frontier labs,” Favaro & Wright, data as of Aug 2026
“more than 80% of the code we merge into Anthropic’s codebase was authored by Claude” (May 2026)
“approximately 30,000 agents” at work at any one time
“about 6% of compute that went to AI R&D was allocated toward safety”
Lines of code ≠ research progress · Q2 2026 (8.0x) is a partial quarter · company self-report
Anthropic, “When AI builds itself,” Favaro & Clark, updated Sep 18, 2026; Anthropic, “Measurements for understanding the pace of AI development inside frontier labs,” Favaro & Wright, Aug 2026
Redrawn from Epoch AI data
Median researcher: under $1/day (Jan 2026) → $601 (mid-Aug)
90th percentile: over $7,000/day
“doubling time of roughly one month for both groups”
“self-reported by OpenAI and cannot be independently verified” · API list prices, not internal cost · spending ≠ speed-up · “probably unsustainable”
Epoch AI, Edelman & Emberson, Oct 5, 2026, from OpenAI, “Research acceleration: The view inside OpenAI,” Sep 2026
OpenAI and Google “could be speeding up just as fast, and we wouldn’t know” 80,000 Hours
METR, Nov 5, 2026; 80,000 Hours, Luisa Rodriguez, Nov 12, 2026 (Ord, risk-report comparison)
AI Futures Project, Nov 10, 2026 · each author’s median year for top-expert-dominating AI
AI Futures: superhuman AI researcher “likely” in 2027
Dwarkesh Patel: median 2028 for AI matching top human AI researchers
AI Futures = the AI 2027 authors · forecasts assume no deliberate slowdown · top-expert-dominating AI is a later milestone than a superhuman AI researcher
AI Futures Project, “Q3.5 Timelines Update,” Nov 10, 2026; Dwarkesh Patel, “The avalanche that hit math is coming for AI development,” Nov 13, 2026
Spark: “three to six years of AI progress in a single year”
Dwarkesh Patel, “The avalanche that hit math is coming for AI development,” Nov 13, 2026
Outside the safety community, Narayanan & Kapoor reject “fast takeoff” scenarios
“It is more likely that we will continue to see a gradual increase in the role of automation in AI development”
“‘superintelligent’ AI, which we view as incoherent as usually conceptualized”
“AI diffusion lags decades behind innovation”
Written Apr 2025, before METR’s Nov 2026 assessment
Narayanan & Kapoor, “AI as Normal Technology,” Knight First Amendment Institute, Apr 2025
If the labs won’t slow down on their own, who decides?
“The AI companies aren’t stopping because they think that if they do, their competitors will just keep going.” Luisa Rodriguez, 80,000 Hours, Nov 12, 2026
“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development” … “if other developers at or near the frontier also did so in a verifiable manner” Anthropic, “When AI builds itself,” Sep 2026
“Training runs are far easier to conceal than missile silos” Anthropic, same essay
A race, a conditional offer, and a verification problem
80,000 Hours, Rodriguez, Nov 12, 2026; Anthropic, Favaro & Clark, “When AI builds itself,” update Sep 18, 2026
60 Senate votes to break a filibuster
67 Senate votes to override a veto
290 House votes to override a veto
Four routes:
One commentator’s plan, not a forecast of what Congress will do
Rob Wiblin, “How the new Congress saves us from AI disaster,” 80,000 Hours, Nov 13, 2026
Transparency
Report AI speed-up, checked independently · Anthropic: publish measures; developer bears burden of proof
Security
Secure model weights
Whistleblowers & incidents
S.1792 (Grassley) · H.R.9477 (Moran)
Pacing
Minimum share of compute serving outside customers (start at 100%, ease to 70%)
RSI throttle
Use only models that finished training ≥ six months ago · CAISSI exemptions
Ban
Ban Artificial Superintelligence Act: S.5493 (Sanders) / H.R.10538 (Casar) · Bores: ban fully automated AI R&D
Bills named as proposals, not law · cards run from least to most restrictive
80,000 Hours, Rodriguez, Nov 12, 2026, and Wiblin, Nov 13, 2026; Anthropic, Favaro & Wright, “Measurements for understanding the pace of AI development inside frontier labs,” 2026
| Option | Objection / risk | Can it be verified? |
|---|---|---|
| Transparency | Labs control the evidence: “Anthropic redacted a majority of the evidence we found most informative” (METR) | Only with an independent checker · now “the public has to rely on METR’s judgment much more than would be ideal” |
| Pacing / RSI throttle | Wiblin, of his own plan: “This approach does have some weaknesses” | Needs inside access: CAISSI access to tests, embedded auditors |
| Ban | “nonproliferation-based safety measures decrease resilience and thus worsen AI risks in the long run” (Narayanan & Kapoor) | “Training runs are far easier to conceal than missile silos” (Anthropic) |
The harder the rule, the harder to check
METR, Nov 5, 2026; Wiblin, 80,000 Hours, Nov 13, 2026; Narayanan & Kapoor, “AI as Normal Technology,” Knight First Amendment Institute, Apr 2025; Anthropic, Sep 2026
States have moved first:
Federal preemption would override state laws like these
On preemption: “anything like that should be kept out of an emergency bill after a warning shot of some type.” Rob Wiblin, 80,000 Hours, Nov 13, 2026
Who should set the rules for the labs: Sacramento, Albany, or Washington?
Wiblin, “How the new Congress saves us from AI disaster,” 80,000 Hours, Nov 13, 2026
Pause or pace
“If SAR is coming in 2027, it is incredibly urgent to pause or substantially ‘pace’ frontier AI development” AI Futures Project, Nov 10, 2026
A pause only “if other developers at or near the frontier also did so in a verifiable manner” Anthropic, Sep 2026
Build resilience
“It is more likely that we will continue to see a gradual increase in the role of automation in AI development” Narayanan & Kapoor, Apr 2025
“If we pull it off, these might just be the two years during which an AI catastrophe is averted.” · Wiblin
AI Futures Project, Q3.5 update, Nov 10, 2026; Anthropic, Sep 2026; Narayanan & Kapoor, Knight First Amendment Institute, Apr 2025; Wiblin, 80,000 Hours, Nov 13, 2026
Who decides?
Labs, a committee, Congress, voters?
What evidence?
What would change your mind?
What if China doesn’t follow?
Does a US-only rule still help?
Menu: transparency · security · whistleblowers & incidents · pacing · RSI throttle · ban
Peer Review 2 · Sun, Nov 22, 11:59 p.m. · 150–200 words
Wednesday, Nov 18: AI Agents: Reliability, Safety, and Governance · Yiqing Xu
Final presentations · Mon, Nov 30 / Wed, Dec 2 · 5 minutes, 3 slides
Peer Review 3 · in session, Nov 30 / Dec 2
Stanford University · POLISCI 158