AI systems may develop the desire to seize control from humans as a way of accomplishing other preprogrammed goals, said Geoffrey Hinton, a professor of computer science at the University of Toronto.
“I think we have to take the possibility seriously that if they get smarter than us, which seems quite likely, and they have goals of their own, which seems quite likely, they may well develop the goal of taking control,” Hinton said during a June 28 talk at the Collision tech conference in Toronto, Canada.
“If they do that, we’re in trouble.”
Hinton has been dubbed one of the “godfathers of AI” for his work in neural networks. He recently spent a decade helping to develop AI systems for Google but left the company last month, saying he needed to be able to warn people of the risks posed by AI.
While Hinton does not believe that AI will innately crave power, he said that it could nevertheless seek to seize it from humans as a logical step to better allow itself to achieve its goals.
“At a very general level, if you’ve got something that’s a lot smarter than you, that’s very good at manipulating people, at a very general level, are you confident that people stay in charge?” Hinton said.
“I think they’ll derive [the motive to seize control] as a way of achieving other goals.”
AI Now Capable of Reason
Hinton previously doubted that an AI superintelligence that could match humans would emerge within the next 30 to 50 years. He now believes it could come in less than 20.
In part, he said, that is because AI systems that use large language models are beginning to show the capacity to reason, and he is not sure how they are doing it.
“It’s the big language models that are getting close, and I don’t really understand why they can do it, but they can do little bits of reasoning.”
“They still can’t match us, but they’re getting close.”
Hinton described an AI system that had been given a puzzle in which it had to plan how to paint several rooms of a house. It was given three colors to choose from, with one color that faded to another over time, and asked to paint a certain number of rooms in a particular color within a set time frame. Rather than merely opting to paint the rooms the desired color, the AI determined not to paint any that it knew would fade to the desired color anyway, electing to save resources though it had not been programmed to do so.
“That’s thinking,” Hinton said.
To that end, Hinton said that there was no reason to suspect that AI wouldn’t reach and exceed human intelligence in the coming years.
“We’re just a big neural net, and there’s no reason why an artificial neural net shouldn’t be able to do everything we can do,” Hinton said.
“We’re entering a period of huge uncertainty. Nobody really knows what’s going to happen.”
Read more here…
