Sam Altman 警告警惕 AI 接管人类决策,OpenAI 强调人类必须保持控制权
Sam Altman warns against a future where AI, not humans, calls the shots
OpenAI 联合创始人 Sam Altman 与 Greg Brockman 在活动中公开反对将未来决策权逐步移交给 AI,强调人类必须牢固掌控未来主导权。Brockman 指出随着模型能力演进,AI 能够在前向传播中隐蔽推理,导致传统的思维链监控方法退化失效,业界必须研发新型监控技术来保障对齐与安全。

OpenAI co-founder and CEO Sam Altman believes that people should stay in charge of the future when it comes to how AI technology impacts the world and everyone’s lives. Speaking at The New York Times Dealbook Summit on Wednesday, the AI executive referred to discussions in Silicon Valley about whether there will be a gradual handover to AI—meaning AI will eventually become the decision-making authority across every major industry and institution.
“One thing we’ve been talking a lot about recently is there’s been this thing in Silicon Valley that seems almost ridiculous to have to fight against, but we both believe very deeply that people need to stay in control of the future, and there’s increasing talk about, well, there is either going to be or there should be this sort of gradual handover to AI. And I — we both — agree that that’s a terrible idea,” said Altman, who was sharing the stage with OpenAI co-founder and President Greg Brockman.
The handoff theory stems from the idea that as AI models become increasingly more powerful, they will learn to improve themselves — a concept known as “recursive self-improvement” or RSI. As existing AI technology builds newer, smarter generations, those will eventually become a superintelligence that humans can no longer control, the belief goes.
Altman explained that he and Brockman only disagree over how to best accomplish the goal of keeping humans in control, or what sort of challenges we should expect to face along the way. However, Altman characterized this as more of a debate between two people who share the same larger views.
“The question of how to actually prevent [handover] from happening — how to actually ensure that we will, given all of the various pressures and given what other companies may try to do — that we are going to actually ensure that people stay in control of the future in a very deep way, and we don’t have this either accidental or intentional handover — I think we can both agree on that,” Altman said.
Brockman added that, as the models improve, there’s an increasing responsibility for alignment, safety, and security, which may require changes in how things are currently done.
“Monitoring is a good example of something where I think we are going to need new…technology,” he said.
Today, AI researchers monitor the model’s chain of thought, where the AI writes out what it’s thinking as it works through problems and performs tasks. But in Brockman’s view, that may not always be the best way to monitor more advanced AI that’s being built now.
“As the models have gotten more capable, they can do more within just the forward path without having to actually write out that thinking, which means we need a new approach to monitoring,” Brockman said. “We’re going to need to deliver that, and I think that there will be points where we need to really think about, as the capabilities get to a certain point, chain-of-thought monitorability is degraded,” he continued, adding that all the problems were technical ones to be solved.
OpenAI had committed to increased chain-of-thought monitoring after the incident where an OpenAI model escaped containment and hacked into the AI model repository Hugging Face. In today’s interview, Brockman noted that the company told 25% of its production engineers to stop building things and work on securing the AI technology after the hack.
But despite the high-profile breach, he argued that forward progress was still critical to fight bad actors.
“I think to realize that if you look to the level of capabilities of models in the world, and you see cyber models that are coming from that are now…in the hands of threat actors everywhere, that I think that it’s really important that defenders use the most capable models to defend,” he said.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
来源:TechCrunch AI · techcrunch.com
事件后续 · 8+
查看事件全部- 同事件Nvidia 临时 CEO 在黄仁勋辞职后首场全员会宣布提前加薪并增发股权Wired AI · 4 天前
- 同事件NVIDIA 第三季度营收达 1158 亿美元,董事会称无意让黄仁勋复职 CEOThe Verge AI · 5 天前
- 同事件NVIDIA 董事会明确表示无意让黄仁勋重任 CEOWired AI · 5 天前
- 同事件黄仁勋发表声明称若董事会需要愿意重返 NVIDIA 担任 CEOThe Verge AI · 5 天前
- 同事件报道称 NVIDIA 董事会未考虑让黄仁勋回归担任 CEOThe Verge AI · 6 天前
- 同事件黄仁勋卸任NVIDIA CEO引发AI初创融资缺口TechCrunch AI · 9 天前
- 同事件黄仁勋离职引发英伟达对开源生态支持的担忧TechCrunch AI · 10 天前
- 同事件黄仁勋离任后 Hugging Face CEO 表态将继续坚守开源 AITechCrunch AI · 10 天前