Sign In

Communications of the ACM

ACM TechNews

Google Wants to Challenge AI with 200 Tasks, to Replace the Turing Test

View as: Print Mobile App Share:

The researchers found that although the results achieved by AI models rose as they gained more computing power, AIs still performed poorly on most tasks compared with humans.

Credit: Cosminxp Cosmin/Getty Images

Artificial intelligence (AI) researchers at 132 institutions, including Google, have proposed an update to the Turing test for rating the capabilities of AI technology.

The researchers suggested using the Beyond the Imitation Game benchmark (BIG-bench), comprised of 204 diverse tasks designed not to be fully solved by state-of-the-art AI models.

The researchers observed that AI models performed poorly on most of the tasks compared with humans, although they showed improvement as computing power increased.

Adrian Hilton of the U.K.'s University of Surrey said, "I can see that setting up a set of benchmarks is one way of comparing one machine-learning algorithm with another, one AI with another. But I don't think that necessarily answers the question of intelligence though."

From New Scientist
View Full Article - May Require Paid Subscription


Abstracts Copyright © 2022 SmithBucklin, Washington, DC, USA


No entries found

Sign In for Full Access
» Forgot Password? » Create an ACM Web Account