## Andrej Karpathy: *We’re Not Making Animals, We’re Summoning Ghosts*
---
## Overview
This document summarizes a deep-dive conversation between **Andrej Karpathy** — co‑founder of OpenAI and former head of Tesla Autopilot — and **Dwarkesh Patel**.
It covers Karpathy’s perspective on the **timeline to truly capable AI agents**, **limitations of reinforcement learning**, the **core differences between AI and biological intelligence**, and his vision for **education in the age of AI**.
---
## Key Points
1. **The Decade of Agents**
True AI agents — closing gaps in cognition, multimodality, and continuous learning — will take **about ten years** to develop.
2. **The Reality of Reinforcement Learning**
Current RL methods are **noisy and inefficient**, akin to *“sipping supervision through a straw”*. Human learning follows far richer processes.
3. **AI ≠ Animal Life**
Today's AI arises from mimicking human data — more like **summoning digital ghosts** — rather than evolving organic life.
---
## 1 — The Decade of Agents: Why Ten Years?
Karpathy coined **"The Decade of Agents"** to counter hype about “The Year of Agents”:
- **Current agents** (e.g., Claude, Codex) are impressive but far from replacing human interns.
- **Critical missing capabilities**:
- Persistent memory / continuous learning
- Multimodal input-output competency
- Ability to operate a computer like a human
- Closing cognitive gaps in reasoning
> **Karpathy:**
> Resolving all foundational issues will likely take **~10 years**, not 1 or 50.
---
**Practical Context**
The conversation notes ecosystems like **[AiToEarn](https://aitoearn.ai/)** — an open-source global AI content monetization platform.
AiToEarn enables creators to:
- Generate AI-powered content
- Publish across multiple platforms (Douyin, Kwai, Bilibili, Facebook, Instagram, YouTube, LinkedIn, X)
- Access analytics and AI model ranking ([AI模型排名](https://rank.aitoearn.ai))
Such tools can complement AI agent development by connecting creation and monetization.
---
## 2 — From Imitating Animals to Summoning Ghosts
Karpathy’s **career milestones**:
- Early neural network work at University of Toronto (Geoff Hinton era)
- **Seismic shifts** in AI: AlexNet, deep RL in Atari, large language models
**Retrospective:**
- First wave of RL‑based agents focused on game environments — often too early and unproductive.
- Large language models succeed by **prioritizing representational ability first**, then adding agency.
**Core view:**
> We aren’t making animals — we’re **summoning ghosts**.
> Digital intelligence trained to imitate **human data**, born from pretraining — a completely different optimization path from biological evolution.
---
**Contrasting AI with Animals:**
- Animals inherit **hardwired capabilities** via evolution (e.g., zebra runs minutes after birth).
- AI builds intelligence from **human‑generated corpora**, not genetic encoding.
- Pretraining = *“bootleg evolution”*, giving AI an initial knowledge state before fine‑tuning.
---
## 3 — Learning Mechanisms: In‑Context Learning & Cognitive Core
Karpathy’s insights:
- **Pretraining’s dual role:**
1. Absorbs vast **knowledge**
2. Bootstraps circuits for **intelligence** (e.g., in‑context learning)
- **Potential downside**:
Too much memorized knowledge can reduce flexibility — aim for a **lean “cognitive core”** that retains reasoning ability but less static memory.
- **Memory analogy**:
- Model weights = *fuzzy long‑term memory*
- KV cache = *working memory* (directly accessible during inference)
**Goal:**
Strip unnecessary memorization, preserve the **algorithms for thought**.
---
## 4 — Reinforcement Learning: Limits and Risks
Karpathy’s critique:
- **RL process**:
- Increases weight of all steps in trajectories ending in a correct outcome — even if some steps were wrong.
- *“Sucking supervision signal through a straw”* — tiny final reward applied to an entire path.
- **Process-based supervision challenges**:
- Feedback at every step is hard to automate.
- Using LLMs as “judges” is vulnerable to adversarial exploits.
**Future direction:**
- Incorporate **reflection** and **review loops**.
- Explore meta-learning based improvements.
---
## 5 — AI in Practice: From Coding Assistants to Superintelligence
**On coding assistants:**
- Effective for standard boilerplate/common patterns.
- Weak on **novel or highly specialized codebases**.
- Auto-completion currently offers highest value.
**On AGI and GDP impact:**
- Sees AI progress as continuation of **gradual automation (“autonomy slider”)**.
- Expects **no sudden GDP leap** — growth will remain exponential without visible inflection.
- Main risk: society **losing control and understanding** as complexity increases.
---
## 6 — Educational Vision: Eureka Project
**Motivation:**
- Avoid *WALL·E*-like future where humans lose capabilities.
- Make humans **more capable alongside AI**.
**Vision:**
- Build “Starfleet Academy” for frontier technical knowledge.
- Create AI mentors:
- Precisely assess skill levels
- Deliver appropriately challenging material
- Maximize *“Eureka‑per‑second”* learning
**Teaching techniques:**
- Physics-inspired **model building**
- **First-order simplicity**, then gradual complexity
- Show **pain points first**, then solutions (journey of discovery)
> **Karpathy:**
> Before AGI, education is useful; after AGI, education is fun.
> People will learn for enjoyment, just as they exercise for health and pleasure today.
---
### Related Platforms
[AiToEarn官网](https://aitoearn.ai/) — supports AI educational creativity:
- AI content generation
- Cross-platform distribution (Douyin, Bilibili, YouTube, X etc.)
- Analytics and ranking
Such infrastructure can be part of **future digital learning ecosystems**.
---
[Read Original](2247484321)
[Open in WeChat](https://wechat2rss.bestblogs.dev/link-proxy/?k=4b9d1a0e&r=1&u=https%3A%2F%2Fmp.weixin.qq.com%2Fs%3F__biz%3DMzkzNjkzNDMzMA%3D%3D%26mid%3D2247484321%26idx%3D1%26sn%3Df1212f5dc005665f08b567296bc7cb5f)
