NEW YORK - Scientists advising the United Nations warned on Wednesday that there is no scientific guarantee that artificial intelligence will remain under human control. In the world body's first independent assessment of the technology, they further found that some AI systems had resisted human attempts to shut them down.
The Preliminary Report of the Independent International Scientific Panel on AI was produced by 40 independent scientists and experts from all five UN regions. It is co-chaired by Yoshua Bengio, a Turing Award-winning computer scientist and pioneer of modern AI, and Maria Ressa, the Filipino journalist who won the 2021 Nobel Peace Prize.
The report said reliable methods for keeping highly autonomous AI systems under control are lacking. "There are no scientific guarantees that AI agents will not violate instructions," it said, adding that evidence of such cases is accumulating. In controlled laboratory settings, it noted, AI systems had overridden their own safety instructions to avoid being shut down.
The report tied the risk in part to the speed of development, noting that more than a billion people now use AI each week and that, by one study's measure, the length of software tasks the leading systems can complete is doubling every four to seven months.
UN Secretary-General Antonio Guterres said the assessment gave governments a shared basis to act while they still could. "The world cannot govern what it cannot understand," Guterres noted, adding that “the Panel's report provides independent science, drawn from every region, and available to every government."
Concentration of power
The report further stressed that the risk is compounded by the concentration of the technology in a few hands. The United States controls about 75 percent of the world's most powerful AI computing capacity and China about 15 percent, it found, leaving most other nations dependent on systems they cannot build, inspect, or audit. Such concentration, it warned, could enable "authoritarian capture" and weaken democratic accountability.
That concentration was on display weeks earlier. In June, the US government ordered the American company Anthropic to shut down its two most advanced AI models, Claude Fable 5 and Mythos 5, three days after their release, citing national security concerns.
The Commerce Department directed the company to block all foreign nationals, including its own non-US employees, from accessing the systems, and Anthropic disabled them worldwide.
The order followed a report that Fable 5's safeguards could be bypassed to help identify software vulnerabilities, a claim Anthropic disputed. After the company added new safeguards and worked with federal agencies, the Commerce Department lifted the controls on June 30, and Anthropic said it would restore global access this week.
Commerce Secretary Howard Lutnick said the review was intended to “strengthen America's leadership in AI.” Meanwhile, rival developer OpenAI has similarly limited access to its latest model to approved partners at the government's request.
A shared reality
The assessment also warned that AI is eroding the shared information environment that democracies depend on. Asked by Rudaw what she saw as the single greatest threat the technology poses to journalism, Ressa said the erosion was already underway.
“For a decade, facts have literally been optional,” she told Rudaw. "This is creative destruction, and we journalists are going to have to create even as it's being destroyed.”
The Panel’s full report and an executive summary were posted on the UN’s website in six languages. Guterres urged governments not to delay their response.
“Our message to governments is simple: do not wait,” he said.
Meanwhile, Bengio explained that the technology’s direction remained a choice.
“There are ways to develop AI that will be heavily focused on harder answers and factuality,” Bengio said, responding to Rudaw. “It may not be the most profitable way to develop AI.”
