Stephen Hawking: AI could subdue humans with weapons

Professor Stephen
Hawking.
ANDREW COWIE/AFP/Getty
Images

• Stephen Hawking, who died earlier this year, wrote a
  collection of essays that were released on Tuesday. 
• The book, Brief Answers to the Big Questions,
  includes a chapter on the potential dangers of artificial
  intelligence. 
• Hawking wrote that superhuman intelligence could manipulate
  financial markets, human leaders, and more without our control.
• People should invest more in researching the potential
  effects of artificial intelligence in order to prevent losing
  control of machines, Hawking said.

Machines with superhuman intelligence have the potential to
subdue humans with weapons that “we cannot even understand,”
Stephen Hawking
wrote in a posthumous collection of essays released Tuesday.

The book,
Brief Answers to the Big Questions, comes seven months
after the
world-famous scientist’s death. It features commentary on a
variety of topics, including black holes and time travel, though
some of the most dire predictions relate to artificial
intelligence.

If computers keep doubling in both speed and memory capacity
every 1.5 years, Hawking wrote, they will likely become more
intelligent than people in the next 100 years. Such an
intelligence “explosion” would require us to make sure that
computers do not begin harming people, he said.

“It’s tempting to dismiss the notion of highly intelligence
machines as mere science fiction, but this would be a mistake,
and potentially our worst mistake ever,” Hawking wrote. 

Hawking noted that integrating artificial intelligence with
neuroscience, statistics, and other fields has yielded many
successful inventions — including speech recognition, autonomous
vehicles, and machine translation. One day, even diseases and
poverty could be eradicated with the help of artificial
intelligence, he said.

While technology could benefit humans a great deal, Hawking
wrote, researchers need to focus on making sure we avoid the
risks that come with it. In the near future, artificial
intelligence could increase economic equality and prosperity
through job automation. But one day, the same systems could take
over our financial markets, manipulate our leaders, and control
our weapons, Hawking said.

“Success in creating AI would be the biggest event in human
history,” he wrote. “Unfortunately, it might also be our last,
unless we learn how to avoid the risks.”

Researchers have not focused enough on artificial
intelligence-related issues, Hawking said, though some technology
leaders are stepping in to change that. Hawking cited Bill Gates,
Elon Musk, and Steve Wozniak as examples of people who share his
concerns, adding that awareness of potential risks is growing in
the tech community. 

People should not turn away from exploring artificial
intelligence, Hawking wrote. Human intelligence, after all, is
the product of natural selection in which generations of people
adapted to new circumstances, he said.

“We must not fear change,” Hawking wrote. “We need to make it
work to our advantage.”

When humans invented fire, people struggled with controlling it
until they created the fire extinguisher, Hawking wrote. This
time around, we cannot afford to make mistakes and respond to
them later, he said.

“With more powerful technologies such as nuclear weapons,
synthetic biology and strong artificial intelligence, we should
instead plan ahead and aim to get things right the first,”
Hawking wrote. “It may be the only chance we will get.”