If We Don’t Rely on Sheer Compute Power, How Else Can AI Evolve?

This is the ultimate question Ilya Sutskever brought back after disappearing from the public eye for quite some time. In his first in-depth interview since the founding of SSI, he offered a counterintuitive view: the “Scaling” recipe that has sustained the industry over the past few years — the idea of “more power equals better results” — is no longer as effective.

Original video: https://www.youtube.com/watch?v=aR20FWCCjAs

But this isn’t bad news. In Ilya’s view, we’re entering a more advanced stage. Using analogies like “the learning ability of a 15-year-old” and “ancient genetic evolution locks,” he redefines the path toward AGI — nurturing a brain with “general learning intuition.”

This was an extremely dense interview in terms of information. Below is the full translation.

---

1. Even at the AI Singularity, Life Won’t Feel That Different at First

Ilya Sutskever: Do you know what’s craziest of all? The fact that this is really happening.

Dwarkesh Patel: You mean, what exactly?

Ilya Sutskever: Don’t you think? Everything about AI — everything happening in the Bay Area — it’s like science fiction turning into reality.

Dwarkesh Patel: Actually, another crazy thing is how slow takeoff feels so… ordinary. You’d think that humanity investing 1% of its GDP into AI would be an earth-shattering event. But in reality, it feels… meh.

Ilya Sutskever: It turns out we adapt to new things very fast. And right now, everything still feels abstract. What I mean is — you just see the news: some company announced another astronomically large investment. And that’s it. It doesn’t really hit you personally in any tangible way yet.

Dwarkesh Patel: Why don’t we start from there? I think this is interesting. Your point — that from an ordinary person’s perspective, even at the singularity, life wouldn’t feel that different — I think that might remain true for a long time.

Ilya Sutskever: I don’t think so. When I said “it doesn’t feel that different,” I was referring to “okay, another company announced an unimaginably huge investment number.” People are numb to that; it’s just a number, and they don’t know how to process it. But I believe the impact of AI will eventually be deeply felt. AI will infiltrate the entire economic system, driven by very strong economic incentives, and when that happens, the shock will be obvious.

Dwarkesh Patel: So when do you think that shock will come? Right now, there’s this oddity: models seem smarter than their actual economic impact suggests.

Ilya Sutskever: Right — that’s one of the most baffling aspects of today’s models. It’s hard to reconcile this contradiction: on one hand, they perform incredibly well on various evaluations — you look at a question and think, “that’s a hard one,” and yet the model nails it; on the other hand, the real economic output lags far behind. It’s perplexing — how can a model be brilliant in some areas, and then in others make the same stupid mistake twice in a row?

Take an example: suppose you use so-called vibe coding (coding by feel without deep understanding, assisted by AI) to write a program, and then you hit a bug. You tell the model, “Help me fix this bug.” The model says, “Oh wow, you’re absolutely right, there’s a bug. I’ll fix it immediately.” Then it introduces a second bug. You tell it, “Now you’ve made a new bug.” It says, “Oh no, how could I do that? You’re right again.” And then it reintroduces the original bug. The two of you end up endlessly looping between these two bugs.

---

In this context, the discussion about AI’s capability mismatch to economic reality also raises questions about how creators and businesses can actually capture AI’s value efficiently. Emerging open-source platforms like AiToEarn官网 are attempting to bridge that gap — enabling creators to use AI to generate, publish, and monetize multi-platform content simultaneously across major global and local networks. By connecting AI generation tools, publishing pipelines, analytics, and even AI模型排名, initiatives like AiToEarn illustrate one pathway for translating AI’s raw capability into sustainable economic impact.

How Could This Be?

I'm not entirely sure. But it does suggest that there’s something unusual going on behind the scenes.

I have two possible explanations.

---

1. The “Whimsical” Explanation

Perhaps RL (Reinforcement Learning) training makes a model too single-minded — overly focused on one objective — resulting in a sharper sense in certain dimensions but a diminished capacity for global perception elsewhere.

This kind of excessive, one-track-minded optimization could cause failures in some basic operations despite improvements in targeted areas.

---

2. The More Concrete Explanation

When we were only doing Pre-training, the question of “what data to use” was basically solved — the answer was “all of it.” During pre-training, you want to feed every scrap of data you can find into the model; the more, the better. You don’t need to agonize over “this dataset or that dataset.”

But once you start doing RL training, you’re forced to confront that choice.

Teams will say: “I want to run an RL training targeting capability X, another for capability Y.” From what I know, each company has dedicated groups constantly creating new RL environments and adding them to the training mix.

The question is: what exactly are these environments? The degrees of freedom are immense — you can design RL environments in a million different ways.

---

How Evals Influence RL

One fairly common (and often unintentional) scenario is that people take inspiration from evaluation sets (Evals).

Thinking goes like this: “We want our model to look impressive on release, so how do we design RL training that boosts its score on this specific benchmark?”

I’m certain this is happening. And it explains a lot:

Combine this with the fact that model generalization is still imperfect, and you get a clear picture of why benchmark performance can differ so dramatically from real-world performance.

And the true essence of that gap? We still don’t fully understand it today.

---

Dwarkesh Patel: I like the idea that the real “reward hackers” aren’t the models — they’re the human researchers who obsess over Evals. The problem you’re describing could be understood from two angles:

• If it turns out that “superhuman results in programming competitions” don’t automatically translate into better judgment or taste in real-world codebases, maybe the fix is to expand your environment set — not just test on competitive programming problems, but also on “Can it write good code for scenario X?” “Can it produce something high quality for scenarios Y and Z?”
• Or maybe the real question is — Why assume that becoming superhuman at competitive programming will make a model into a generally better programmer?

Perhaps the right approach isn’t endlessly stacking new environments, but rather finding ways for skills learned in one environment to transfer effectively to other tasks, thus genuinely improving broad capability.

---

Ilya Sutskever: Let me give you an analogy — since we’re talking about competitive programming, let’s stick with that.

Imagine two students:

• Student A decides to become the best competitive programmer in the world. They spend 10,000 hours grinding just this, memorize every problem and proof technique, and can implement algorithms instantly and flawlessly. Eventually, they truly become a top contender.
• Student B thinks competitive programming is interesting and trains for about 100 hours — much less than A — but still performs quite well.

Which one do you think will have a better career in the long run?

---

Dwarkesh Patel: The second one.

Ilya Sutskever: Exactly. Current models are like Student A — maybe even more extreme.

We say: “Let’s make the model excel at competitive programming. Feed it every competitive problem ever made, then augment the dataset and create even more variations, then train it on all of them.”

The result is an extremely proficient “problem-solving model.”

In this analogy, you can easily see why such intense specialization may not generalize well to other tasks.

---

Dwarkesh Patel: In the human analogy, what was Student B doing before their 100-hour “fine-tuning”? And what’s that equivalent for a model?

Ilya Sutskever: That, I think, is what we might call "It" — that spark, that intangible “spirit.” I’ve met people like this during my undergraduate years, so I know they really exist.

---

Final Note:

This discussion of generalization and over-specialization connects to a broader point about how creators — whether in AI or other fields — might aim for balanced skill development instead of over-optimizing for one metric. In the AI world, tools like AiToEarn官网 provide an open-source platform that lets creators use AI to generate, publish, and monetize multi-platform content, while tracking actual real-world engagement rather than just benchmark scores. By simultaneously publishing across major platforms such as Douyin, Kwai, WeChat, Bilibili, Rednote, Facebook, Instagram, LinkedIn, Threads, YouTube, Pinterest, and X, AiToEarn bridges the gap between “evaluation performance” and “real-world impact” — a principle closely related to the very challenge discussed here.

Dwarkesh Patel:

Interestingly, we need to distinguish between this “spiritual essence” and “what exactly pretraining does.”

One way to interpret your point about pretraining data selection is: it’s actually not so different from “10,000 hours of practice,” except you can “package” those 10,000 hours for free inside pretraining, because the content already exists in the data distribution used for it.

But perhaps what you’re hinting is: pretraining doesn’t actually bring the level of generalization we imagine — it just leverages sheer scale of data, and that generalization may not be much more advanced than what RL delivers.

Ilya Sutskever:

The biggest advantages of pretraining are twofold:

First, the amount of data is truly massive;

Second, you don’t need to agonize over “which data to choose” — you take it all.

The data is highly “natural,” encompassing all aspects of human activity: people's thoughts, experiences, and abundant features describing the world. In a sense, it’s a “holographic projection of the world onto text” made by humanity.

The goal of pretraining is to use vast amounts of data to capture this projection.

The reason pretraining is difficult to fully dissect is that we can’t clearly determine how the model is truly leveraging this mass of data. Whenever the model makes a mistake, you start wondering: “Is it because, by chance, a piece of knowledge wasn’t sufficiently supported in the pretraining data?”

But “being supported by pretraining data” is rather a vague statement; I can’t explain it much more precisely. I don’t think there is any exact analogy for “pretraining” that truly exists in the human world.

---

2. Emotions are not baggage — they are the most efficient way humans make decisions

Dwarkesh Patel:

Here are some analogies people make for human “pretraining.” I’d love to hear your perspective.

One analogy: the first 13 to 18 years of a person’s life — during which they may not have any economic output, but their activities help them understand the world.

Another analogy: imagining evolution itself as a continuous 3-billion-year search, ultimately shaping a human life.

Do you think these count as pretraining?

Ilya Sutskever:

I think both have similarities to pretraining — in fact, pretraining tries to play both of these roles at once. But the differences are huge. The volume of pretraining data is astronomical.

For some reason, even if humans consume only a tiny fraction of pretraining data, after 15 years of growing up, although their total knowledge is far less than that of AI, whatever they do know, they understand much more deeply. At that age, humans simply don’t make the kind of basic mistakes that AI sometimes makes.

Another point — you asked whether this might be tied to evolution? Maybe.

In fact, evolution may have an advantage here. Neuroscientists often study the brain by observing patients with brain injuries. Some cases are stranger than you’d imagine.

I once read about a patient whose brain was damaged (likely due to stroke or accident), resulting in complete loss of emotional processing.

This person could no longer feel sadness, anger, or excitement.

Oddly enough, they remained articulate, could solve simple puzzles, and scored well on exams.

But they could not feel emotion anymore.

The result: they became terrible at making any decision.

They might spend hours deciding which socks to wear. Financially, they began making extraordinarily poor choices.

What does this tell us?

It shows that our innate emotions play a crucial role in making us competent “agents.”

Back to pretraining — if one could fully extract its value, perhaps a similar effect could be achieved. But… hmm… hard to say whether pretraining can really do that.

Dwarkesh Patel:

What is that “thing”? … Clearly, it’s not just emotions. It seems to resemble a Value Function, telling you the ultimate reward for any decision.

Don’t you think such a thing is implicitly included in pretraining?

Ilya Sutskever:

Possibly. I’m just saying it’s not 100% certain.

Dwarkesh Patel:

How do you see emotions? What’s the analogue to emotions in machine learning?

Ilya Sutskever:

They should be considered a Value Function.

But I think there isn’t yet a particularly fitting ML analogy, because in current practice, value functions don’t play such a central role.

Right now, reinforcement learning (RL) methods are quite simplistic: you have a neural network, you give it a problem, and you say “go solve it.”

The model takes thousands upon thousands of actions or thought steps, then generates a solution.

You grade that solution.

This score is then used as a signal to train every action in that entire trajectory you just took.

That means, if you’re working on a long-horizon task — before you find the ultimate solution, there is simply no learning signal during the intermediate process.

This is the simplest RL setup (systems like o1 and R1 seem to be doing roughly this).

---

Note: For AI creators and researchers exploring pretraining, reinforcement learning, and broader agent behaviors, global open-source platforms like AiToEarn官网 provide integrated tools to generate, publish, and monetize AI content across multiple platforms simultaneously — from Douyin and Bilibili to X (Twitter) and LinkedIn. AiToEarn also connects analytics, cross-platform publishing, and AI model ranking (AI模型排名), empowering creators to efficiently translate AI insights into revenue and reach. This aligns closely with the discussion here — understanding “value functions” and efficient decision-making is just as critical in scaling AI creativity as in agent design.

The meaning of a value function is essentially: “I might be able to tell you halfway through whether you’re doing well or poorly.”

Take chess for example — you lose a piece. You don’t have to wait until the game ends to realize that was a bad move; you know instantly at that moment: “I messed up.”

The value function allows you to short-circuit decision-making — you don’t have to wait until the end to act.

Suppose you’re doing a mathematical derivation or writing code, exploring in a particular direction. After a thousand steps of reasoning, you conclude it’s a dead end. The moment you reach that conclusion, you should get a reward signal and backpropagate it all the way to the moment you originally decided to go down that path a thousand steps ago.

That is — you tell yourself long before finding a true solution: “Next time you face a similar situation, avoid going down this road.”

---

Dwarkesh Patel:

DeepSeek R1’s paper mentioned this — the trajectory space is too large, making it hard to learn the mapping between “intermediate trajectories” and “final value.” And in programming, it’s common to start with a wrong idea and then go back to correct it.

Ilya Sutskever:

That sounds like a lack of faith in deep learning. Of course, it might be difficult, but there’s nothing deep learning can’t do. I expect value functions to be extremely useful, and I fully believe they will be reused in the future, even if they’re still immature now.

The case I mentioned earlier of someone with damaged emotional centers might suggest that human value systems are to some degree regulated by emotion — and that regulation has been hard-coded by evolution. Perhaps this is crucial for humans to function effectively in society.

Dwarkesh Patel:

That’s exactly my next question. Regarding the emotional component in value functions, there’s something interesting — they’re both practical and easy to understand.

Ilya Sutskever:

Agreed. Compared to the AI we’re building, emotions are relatively simple. Maybe simple enough that we could depict them in a way humans can understand.

In terms of utility, there’s a “complexity vs. robustness” trade-off: complex things might be highly useful in certain scenarios, but simpler things are often more robust in broader situations.

We can think of it like this: emotions mainly originate from our mammalian ancestors, then received slight fine-tuning during our evolution into early humans. Precisely because they are not overly complex, they still guide us effectively in today’s world, so vastly different from the ancient environment.

Of course, they can make mistakes — for instance, in today’s age of food abundance, our intuitive hunger drive is not just useless, but misleads us.

---

3. Farewell to “Brute-Force Scaling”: Back to the Age of Ideas

Dwarkesh Patel:

People have been talking about scaling data, parameters, and compute. Is there a more general expansion concept? Are there other dimensions we can scale?

Ilya Sutskever:

I think there’s a perspective here that hits the mark. In earlier machine learning (ML), people basically relied on constant tinkering: try this, try that, see if it works — that was the early pattern.

Later, scaling emerged as a key insight. Scaling laws, GPT-3… and suddenly everyone realized: we need to make things bigger.

This is a great example of “how language influences thought.” “Scaling” is just a word, yet it’s magical because it gives a direct action guide — “keep making it bigger.”

And then the question became — what should we scale? Pre-training turned out to be the perfect candidate. It’s a very straightforward recipe. The biggest breakthrough in pre-training was proving that this recipe is a sure win: if you pump a certain amount of data and compute into a sufficiently large neural network, you get good results. Naturally, you then believe: keep scaling, and performance will keep improving.

This method has the advantage that companies are willing to spend money on it — because it’s a low-risk investment. In contrast, putting resources into pure research is much harder, because research means uncertainty, while pre-training almost guarantees returns.

But pre-training eventually hits a hard limit: data is finite. What then? Either find new “reinforced pre-training” methods, or explore reinforcement learning, or entirely different approaches.

Once compute becomes enormous enough, in many ways we circle back to the “research era.”

---

In this context of evolving AI approaches, platforms like AiToEarn are reshaping how creative work scales across systems. By connecting AI content generation, cross-platform publishing, analytics, and model ranking in one open-source ecosystem, AiToEarn helps creators shift from mere scaling to strategic innovation — publishing seamlessly to Douyin, Kwai, WeChat, Bilibili, Rednote (Xiaohongshu), Facebook, Instagram, LinkedIn, Threads, YouTube, Pinterest, and X (Twitter). When data and compute ceilings appear, tools enabling creative breakthroughs become the new frontier.

如果要划时代：2012—2020 是研究时代； 2020—2025 是规模化时代。 这几年大家几乎都在喊“继续扩大！再扩大！”。但当规模已经这么大时，你真的相信再扩大 100 倍就能彻底改变一切吗？ 会有变化，但我不认为仅靠更大规模就能带来根本性的转折。 我们正重新回到研究时代，只不过这一次，我们手里握着的是超级计算机。

Dwarkesh Patel：你刚才说到一个概念——“配方”。那我们现在究竟在扩展什么？ 在预训练里，数据、算力、参数量之间存在像物理定律那样明确的 Power Law 关系。那现在的“新配方”里，这种关系是什么？

Ilya Sutskever：我们已经看到了 Scaling 路径的迁移：从预训练转向强化学习（RL）。 如今大家正在扩展的是 RL。从外界讨论看，近期 RL 消耗的算力可能已经超过了预训练，因为 RL 天生“烧算力”——它需要极长的推理过程（Rollouts），而每次迭代的学习增益又很小。 我甚至不愿意把它称为“扩展（Scaling）”。我更愿意问的是：“你的做法是最有效率的吗？你能不能找到更经济的方式去利用算力？” 这就回到之前提到的价值函数。如果人们真的掌握价值函数，也许资源利用效率能提高很多。 但当你提出一个全新的训练方法时，界限就模糊了：“这到底是 Scaling，还是单纯的科研探索？” 从某种意义上说，我们正在回到从前那种模式：“试试这个，再试试那个……哦，这个有点意思。”

四、人类学车只要10小时，为什么 AI 却要练几亿次？

Dwarkesh Patel：所以问题的核心在于泛化（Generalization）。这里其实包含两个子问题。 第一是样本效率（Sample Efficiency）：为什么模型需要比人类多得多的数据才能学会一项能力？ 第二是意图传递：即使不谈数据量，为什么让模型真正理解“我们想要它做什么”，比让人类理解要困难得多？

对人类来说，学习并不依赖这种死板的、可验证的奖励信号。 比如，你现在肯定在带一群研究员。你跟他们交流、展示代码、解释你的思路，他们就能从中学会如何做研究。 你并不需要为他们设计一套繁琐的人工流程（schleppy bespoke process），比如设立一个个打分点：“做得好，这是下一章课程”、“这一轮训练不太稳定，扣分”。 这两个问题或许是相关的，但我想分别讨论：第二个更像“持续学习”，第一个就是纯粹的“样本效率”。

Ilya Sutskever：关于人类样本效率极高这一点，最可能的解释之一就是进化。 进化在视觉、听觉、运动（Locomotion）这些核心能力上，为我们提供了极少量但最关键的“先验知识”（Priors）。

例如，人类的灵巧度远超机器人。 即便机器人在模拟环境中可以通过海量训练变得灵巧，但要让机器人在现实世界里像人一样、上手就能掌握一项新技能，几乎是不可能的。 你可以说：“哦，因为移动能力对我们的祖先来说太重要了，这种生存压力持续了数百万年，所以我们的神经系统里硬编码了某种不可思议的进化先验。” 视觉也是同理。Yann LeCun 曾说，孩子练十几个小时就能学会开车。确实如此——但那是因为孩子的视觉系统本身已经极其强大了。 我记得我五岁时对汽车非常着迷。我可以肯定，那时我的视觉识别能力已经足够支撑驾驶了。但五岁孩子摄入的数据量其实很有限，大部分时间都只待在父母身边，数据多样性很低。 这说明视觉能力可能深深植根于进化，而非完全靠后天数据堆砌。

但是，当我们谈到语言、数学、编程时，情况就不同了。这些能力出现得太晚，不太可能是进化预装的。

Dwarkesh Patel：但即使在这些“近期才出现的能力”上，人类似乎依然比模型强。模型虽然在做题分数上已经优于普通人类，但它们在学习新东西的能力上真的更好吗？

Ilya Sutskever：问得好。 语言、数学、编程——尤其是数学和编程——恰恰证明了：人类擅长学习，可能并不完全依赖复杂的进化先验，而是因为我们拥有某种更基础、更底层的“通用学习能力”。

逻辑是这样的： 如果某项能力（如走路）对祖先很重要，那我们做得好归功于进化先验。 但如果人类在一些“近期才被发明的能力”上（如写代码）依然表现出极强的学习效率和鲁棒性，那这就证明：人类天生就拥有一种“通用的、极其高效的机器学习算法”。 换句话说：如果连不靠进化积累的领域，人类依然能快速学会，那关键就不在先验知识，而在我们大脑的学习机制本身。

Dwarkesh Patel：那我们该如何理解这种机制？ 青少年学开车，并不是通过外界给的一个“奖励分数”来学习的，而是通过与环境的互动。样本量很低，却能快速掌握，而且极度鲁棒。有没有机器学习的类比？

Ilya Sutskever：你问青少年司机如何在没有外部教练时刻打分的情况下自我纠正。答案在于：他们自带价值函数。人类有一种极其强大的普遍感知能力。无论这个内在的价值体系是什么——除了一些成瘾行为会导致短路外——它在绝大多数情况下都是非常稳固的。

所以，当一个青少年坐上驾驶座，他不需要别人告诉他，他立刻就能感觉到自己开得稳不稳、哪里处理得不好。 他们拥有即时的内在反馈（Internal Feedback）。再加上年轻人本身极快的学习速度，十个小时后，他们自然就成了老司机。

Dwarkesh Patel：我好奇的是，这到底是怎么做到的？为什么对我们来说这么自然，对模型来说却这么难？我们需要怎样重新构思训练方式，才能逼近这种能力？

Ilya Sutskever：这是一个非常好的问题，我对此也有很多想法。 但是，很遗憾，我们现在处在一个并非所有机器学习理念都能公开讨论的时代。而这正是那些无法轻易公开讨论的核心机密之一。 我相信是有路径可以做到的。人类的存在本身就证明了这种方法的可行性。

当然，可能还存在另一个变量：人类神经元的实际计算能力，可能比我们目前建模的要强很多。如果这是事实，那么我们要模拟它会比想象中更难。 但无论如何，我相信这确实指向某种机器学习的根本原理。只是出于某些原因，我无法在这里展开。

五、历史证明：伟大的创新，往往不是靠“烧钱”烧出来的

Dwarkesh Patel：我很好奇。如果你认为我们已经重新进入“科研时代”，那么作为亲历过 2012–2020 那段黄金时期的人，你觉得现在的科研氛围会变成什么样？ 毕竟，即使在 AlexNet 之后，实验所需的算力也是指数级增长的。现在的“科研时代”是否依然需要庞大的计算资源？还是说我们需要去故纸堆里翻旧论文？

Ilya Sutskever：“规模化时代”（Scaling Era）的一个后果是：规模本身抽干了房间里所有的空气。由于“扩大规模”被证明太有效了，所有人都去做同一件事，最后变成了一个“公司数量远多于创意数量”的局面。 硅谷有句老话：“创意不值钱，执行力才是一切。” 这话在大方向上没错。 但后来我在推特上看到一句反讽：“如果创意这么廉价，那为什么现在没几个人有创意？” 我觉得这话说得更准。

如果你从“瓶颈”的角度看科研历史： 上世纪 90 年代，许多研究者其实有很好的想法，但受限于算力，只能做玩具级的演示，说服不了任何人。那时的瓶颈是算力。 而在规模化时代，算力暴涨，瓶颈转移了。

这并不意味着现代科研一定要用到极限规模的算力。 举个例子：AlexNet 当年只用了两块 GPU。Transformer 刚问世时的实验，大多也就是在 8 到 64 块 GPU 上跑出来的。按今天的标准，那简直就是几块游戏卡的水平。 没有哪篇奠基性的论文是靠动用整个数据中心才写出来的。 当然，如果你要构建一个“最强系统”，更多算力肯定有帮助——尤其当大家都只有同一把锤子（Scaling）的时候，锤子的大小就成了唯一的护城河。 但科研本身？并不需要无限制的大规模计算。

Dwarkesh Patel：我问这些，是因为你当时就在现场。 Transformer 刚提出时并没有马上爆红。它后来成为行业标准，是因为人们发现它在更大的算力上能持续扩展。 那假设 SSI 现在有 50 个不同方向的想法，在没有其他顶尖大厂那种“无限算力”的情况下，你们怎么判断哪个是下一个 Transformer，哪个是死胡同？

Ilya Sutskever：这里我可以澄清一下。 其实 SSI 用于纯研究的算力，比外界想象的要多得多。 简单的算术题：SSI 虽然“只有”几十亿美元融资，但你要注意，大厂那些巨额的算力预算，绝大部分是被推理（Inference） 吃掉的——也就是服务用户。 其次，大厂为了维持产品，需要养庞大的工程团队、销售团队，研究资源会被各种产品需求稀释。 反观 SSI，我们的钱几乎全部砸在研究上。 更重要的是：如果你在做真正“与众不同”的研究，你真的需要把规模拉满才能证明它是对的吗？我不这么认为。对于验证我们的方向，目前的算力完全足够让我们说服自己。

Dwarkesh Patel：那 SSI 未来怎么赚钱？

Ilya Sutskever：目前我们只专注于研究。 商业化的问题，等到技术突破了，答案自然会显现。

六、真正的超级智能不是“全知全能的神”，而是一个“15岁的少年”

Dwarkesh Patel：SSI 的计划仍然是直通超级智能（Straight shot to Superintelligence）吗？

Ilya Sutskever：也许吧。这个策略确实有它的道理——远离市场的喧嚣（Rat race）是一件好事。 这能让你避免因短期商业竞争而做出妥协。 但在两种情况下我们可能会调整策略： 第一，实际研发时间比预期的长； 第二，我认为让世界尽早看到强大的 AI，本身就是一种巨大的价值。

Dwarkesh Patel：为什么“直通”会是默认选项？ OpenAI、Anthropic 都强调“迭代部署”，让公众慢慢适应。为什么你觉得闷头憋大招反而更好？

Ilya Sutskever：正反两方面都有道理。 支持“直通”的理由是：一旦卷入市场，你就不得不面对艰难的权衡，动作容易变形。 但在我看来，“展示 AI”的价值被低估了。 你写一篇论文、发一篇博客，说“AI 未来会如何如何”，大家看完点个头就忘了。 但如果你让大家亲眼看到 AI 能做什么，那种冲击力是完全不同的。只有真正接触到实体，人类社会才能真正理解我们要面对的是什么。

Dwarkesh Patel：我同意。而且不仅仅是“理解”，更是为了安全。 航空业之所以安全，是因为飞机每天都在飞，事故被发现、被修复，系统才越来越稳健。Linux 之所以稳健，是因为全世界都在用。 我不确定 AGI 为什么要成为例外。超级智能的风险远不止“造回形针毁灭世界”这种科幻情节，更多的是我们根本不知道人类会怎么用它，以及它会如何重塑社会。 逐步普及似乎是让社会产生免疫力的更安全方式。

Ilya Sutskever：我认为，即使采取“直通”路线，发布过程也必然是循序渐进的。关键在于你迈出门的第一步是什么。

另外，你比其他人更强调“持续学习”（Continuous Learning），这非常关键。 我想用一个例子说明“语言如何锁定思维”。有两个词几乎定义了整个行业的认知：AGI 和 预训练。

先说 AGI。这个词是对“狭义 AI”（Deep Blue, AlphaGo）的反动。人们想要一个“通用的”东西，而不是只会下棋的白痴天才。 再说预训练。它之所以流行，是因为它确实带来了一种类似通用的能力。 但这导致了一个误区：我们试图把 AGI 做成一个“成品”。 但如果你仔细想，人类并不是 AGI。人类虽然有基础能力，但我们的知识储备其实很有限。我们真正强的是持续学习。

因此，当我们设想“创造安全的超级智能”时，关键不在于它出厂时“已经掌握了多少技能”，而在于：它在持续学习的曲线上处于哪个阶段？想象一个绝顶聪明、求知欲旺盛的 15 岁少年。他现在懂得不多，但他学习能力极强。 如果你部署这样一个系统，它不是作为一个全知全能的神降临，而是作为一个学习者进入社会。它会经历学习、试错、成长的过程。

Dwarkesh Patel：明白了。你定义的超级智能，不是一个“已经学会所有工作”的系统，而是一个“能学会任何工作”的可成长心智。 这就引出了两种可能： 第一，这个学习算法强到在研发能力上超过了你，于是它开始递归自我进化（Recursive Self-improvement），瞬间起飞。 第二，即便没有发生递归进化，只要你把这个模型复制成千上万份，让它们在全球不同岗位上工作、学习，然后把所有经验合并（Merge） 回一个大脑——这本身就是一种“功能性超级智能”。 你预期这会引发某种形式的“智能爆炸”吗？

Ilya Sutskever：我认为极有可能看到经济的爆发式增长。 业内有两种观点。一种认为只要监管不拦着，经济会疯狂吸纳这些 AI 劳动力。 另一种观点认为现实世界的复杂性（法律、物理限制）会拖慢这一进程。 但我倾向于认为：AI 的劳动效率极高，只要规模铺开，经济增长会非常惊人。不同国家可能会因为监管松紧不同，出现巨大的增长分化。

七、我们必须教会 AI “关爱众生”

Dwarkesh Patel：在我看来，这是一个极其危险的局面。 理论上这完全可能发生：如果一个系统既具备接近人类的学习效率，又能以人类无法做到的方式“融合多个大脑实例”，那它的潜力将远超任何生命形式。 如果它真的强大到能建造戴森球，那带来的经济增长将是指数级的。 所以关键问题是：SSI 凭什么认为自己有能力安全地掌控这种力量？你们的计划到底是什么？

Ilya Sutskever：我的思维方式确实发生了一些变化。我现在更强调 AI 的逐步部署与提前规划。 AI 的核心难题在于：我们讨论的是尚不存在的系统，很难真正想象它会是什么样子。 就像你很难向一个年轻人解释“变老”是什么感觉——你可以尝试描述，但他如果不亲历，就永远无法感同身受。

围绕 AGI 的很多争议，本质上都源于这种想象力的缺失。 AI 和 AGI 的核心问题究竟是什么？就是力量（Power）。 当这种力量真正出现时会发生什么？ 我的结论是：如果大众难以想象，那就必须把它展示出来。

我认为，随着 AI 变得越来越强大，人类的行为会被迫改变。 第一，前沿公司与政府会开始合作。我们已经看到 OpenAI 和 Anthropic 在安全上的联动，这在几年前是不可想象的。随着 AI 力量的显现，政府和公众会强烈要求介入。 第二，只有当 AI 真的显露力量时，安全观念才会发生质变。目前许多人觉得 AI 还是个“傻瓜”，因为它还在犯错。但等到它展示出真正的肌肉时，所有 AI 公司的安全红线都会收紧。这种谨慎现在还没出现，是因为大家还没被吓到。

第三，企业究竟该构建什么？ 长期以来，业界都执迷于“能自我改进的 AI”。 但我认为，有一个更值得构建的目标，未来每个人都会想要它： 那就是：以关怀有感知生命（Sentient Life）为核心的 AI。

我认为，构建一个“关心所有具备感知的生命体”的 AI，比构建一个“只关心人类”的 AI 要容易且自然得多。 原因在于：AI 本身未来也将具备感知能力。 想想镜像神经元（Mirror Neurons）。人类之所以对动物有同理心，是因为我们在用模拟自身的神经回路去模拟对方——这是理解他者最高效的方式。AI 也会演化出类似的机制。

Dwarkesh Patel：但是，即使你让 AI 关心有感知的生物——实际上，如果你解决了阵营问题（Alignment），我并不确定这是否是最佳标准。 未来 AI 的数量将达到数万亿，甚至千万亿。人类在“有感知的生物”中所占的比例将微乎其微。 如果目标是让人类继续主导未来文明，这个标准可能有点“稀释”了人类的权重。

Ilya Sutskever：没错，这可能并不是最完美的标准。我想强调三点：

-

“关爱有感知生命”是一个非常值得纳入考量的安全基线。

-

如果能提前准备好一份“备选方案清单”，供公司在关键时刻参考，将极具价值。

-

能力限制（Capping capability）。如果能对最强大的超级智能施加某种硬性的能力天花板，那能解决很多潜在麻烦。虽然现在我还不知道具体怎么做，但在面对“神级”系统时，这是必须考虑的手段。

八、人类的未来：要么进化成“半AI”，要么彻底沦为旁观者

Dwarkesh Patel：那长期来看，这种平衡如何维持？如果世界上充满了体量堪比大陆的计算智能体，我们该怎么办？

Ilya Sutskever：短期内，如果首批强大的系统能做到“关爱众生”，那局面会保持良性。 但长期会发生什么？ 佛家有云：“唯一不变的是变化本身。”政治结构、社会秩序都有生命周期。现在的稳定结构，过个几十年可能就失效了。

所以从长期看，一种可能的（也许是过于乐观的）模式是：每个人都拥有一个专属的 AI 代理。 它替你赚钱、替你搞政治博弈、替你处理一切，然后定期给你发一份简报：“老板，这是我这周的战果。”你只需点头：“很好，继续。”问题在于：人类完全退出了参与（Out of the loop）。 这是一种高度危险的局面。

我其实并不喜欢下面这个方案，但它在逻辑上确实是一条可行的路径： 那就是人类通过某种类似 Neuralink 的技术，让自己成为“半 AI”。 这样一来，AI 理解的，你也能直接理解；AI 经历的，你也能感同身受。 信息不再通过低带宽的语言传输，而是全息地传递给人类。 在这种情况下，当 AI 处于某种复杂情境中时，人类是真正“在场”的，而不是一个只看简报的旁观者。

九、进化的未解之谜：欲望是如何被硬编码的？

Dwarkesh Patel：我想知道，数百万年前在完全不同的环境中形成的情感，至今仍如此强烈地指导着我们的行为，这是否算是一个“对齐（Alignment）”成功的案例？ 比如，脑干（低级中枢）有一个指令：“去和成功人士交配。” 大脑皮层（高级中枢）负责理解现代社会中“成功”的定义（是金钱？地位？还是才华？）。 但脑干成功地控制了皮层：不管你怎么定义成功，你最终都得听我的去执行。

Ilya Sutskever：这是个深刻的谜题。 进化如何编码像“想吃好吃的”这种低级欲望，很容易理解——嗅觉连接多巴胺，简单粗暴。 但进化如何赋予我们高级的社交欲望？比如渴望被社会尊重、渴望地位。 这些概念在物理世界中并不存在，它们需要大脑进行极复杂的信息处理才能构建出来。 而进化竟然能把这种高维度的抽象欲望，硬编码进我们的基因里。它是怎么做到的？

我曾有过一个猜测（虽然大概率是错的）： 也许大脑皮层的特定功能区（Region）在物理位置上是固定的。 进化可能只是简单地写了一行代码：“当大脑坐标 (X, Y, Z) 的神经元活跃时，释放多巴胺。” 如果这个坐标恰好对应处理“社交信息”的区域，那我们就有了社交欲望。

Dwarkesh Patel：但这个理论有个漏洞：先天失明的人，视觉皮层会被听觉或触觉接管；甚至有人切除了一半大脑，功能区发生了大迁移，但他们依然有正常的社交欲望。

Ilya Sutskever：正是。大脑的可塑性（Plasticity）否定了“固定坐标”的理论。 如果那个理论是真的该多好，事情就简单了。 可惜不是。这依然是一个未解之谜。进化究竟用什么语言在基因里写下了“去追求社会地位”这条指令？我们还不知道。

十、SSI 的差异化与未来推演

Dwarkesh Patel：SSI 的独特之处到底在哪里？其他公司也在做，为什么你觉得你们能成？

Ilya Sutskever：很简单，我有几个关于“理解与泛化”的核心假设，我认为它们是正确的。SSI 就是为了验证这些假设而存在的实验。 我们是一家纯粹的“研究型公司”。 我相信，随着 AI 变得足够强大，所有公司的技术路径最终会趋同（Converge）。 就像登山，到了山顶，路只有那几条。 大家最终都会意识到：必须建立某种可靠的沟通方式，必须确保第一个超级智能是关爱众生的、尊重民主的。

Dwarkesh Patel：你对这类系统（像人类一样学习、最终超越人类）的时间预测是？

Ilya Sutskever：大概 5 到 20 年。

Dwarkesh Patel：既然最终大家会趋同，为什么先发优势不会让一家公司垄断一切？

Ilya Sutskever：历史经验告诉我们，即便一家公司率先突破，其他公司也会迅速跟进，推出类似产品，把价格打下来。 更重要的是专业化分工（Specialization）。 即便有一个通用的学习算法，但一家公司可能在法律 AI 上积累了巨量数据和经验，另一家在医疗 AI 上登峰造极。 竞争总是偏爱专业化。哪怕是在超级智能时代，我也倾向于认为会有多个专精不同领域的巨头并存，而不是一个全知全能的单一霸主。

十一、终极问题——什么是“研究品味”？

Dwarkesh Patel：最后一个问题：什么是“研究品味”（Research Taste）？ 你参与了 AlexNet、GPT-3 等所有历史性突破。你是如何产生这些想法的？

Ilya Sutskever：对我而言，指引我穿越迷雾的北极星，是AI 的“美学”（Aesthetics）。 这意味着要以一种“正确的方式”去思考人类的本质。

举个例子：人工神经元的概念直接源自大脑，这是一个极具美感的想法。 为什么？虽然大脑有沟回、有各种复杂的生化反应，但我们直觉上认为那些只是细节，真正起作用的是海量的连接和局部的学习规则。 再比如分布式表征（Distributed Representation）——大脑不是把记忆存在某个格子里，而是通过连接权重的变化来学习。这不只是模拟，这捕捉到了智能的灵魂。

这种品味，就是倾向于寻找那些简洁、优雅、且符合生物学直觉的方案。丑陋的、拼凑的东西（Ugand/Ugliness）是没有容身之处的。如果一个想法缺乏这种美感，我就不会感到踏实。

这种“自上而下的信念”（Top-down belief）至关重要。 当你做实验时，数据常常会打你的脸。 如果完全依赖数据，你会被一个微小的 Bug 误导，以为这个方向是错的，从而放弃。 但如果你有这种信念，你就会对自己说： “不，逻辑上它必须是对的。肯定是哪里代码写错了，我要把它找出来，而不是换方向。” 正是这种源自对大脑、对数学之美的深刻直觉，支撑着你熬过那些至暗时刻。