Can Machines Imitate Humans? Integrative Turing-like tests for Language and Vision Demonstrate a Narrowing Gap
Authors:
Mengmi Zhang,
Elisa Pavarino,
Xiao Liu,
Giorgia Dellaferrera,
Ankur Sikarwar,
Caishun Chen,
Marcelo Armendariz,
Noga Mudrik,
Prachi Agrawal,
Spandan Madan,
Mranmay Shetty,
Andrei Barbu,
Haochen Yang,
Tanishq Kumar,
Shui'Er Han,
Aman Raj Singh,
Meghna Sadwani,
Stella Dellaferrera,
Michele Pizzochero,
Brandon Tang,
Yew Soon Ong,
Hanspeter Pfister,
Gabriel Kreiman
Abstract:
As AI becomes increasingly embedded in daily life, ascertaining whether an agent is human is critical. We systematically benchmark AI's ability to imitate humans in three language tasks (image captioning, word association, conversation) and three vision tasks (color estimation, object detection, attention prediction), collecting data from 636 humans and 37 AI agents. Next, we conducted 72,191 Turi…
▽ More
As AI becomes increasingly embedded in daily life, ascertaining whether an agent is human is critical. We systematically benchmark AI's ability to imitate humans in three language tasks (image captioning, word association, conversation) and three vision tasks (color estimation, object detection, attention prediction), collecting data from 636 humans and 37 AI agents. Next, we conducted 72,191 Turing-like tests with 1,916 human judges and 10 AI judges. Current AIs are approaching the ability to convincingly impersonate humans and deceive human judges in both language and vision. Even simple AI judges outperformed humans in distinguishing AI from human responses. Imitation ability showed minimal correlation with conventional AI performance metrics, suggesting that passing as human is an important independent evaluation criterion. The large-scale Turing datasets and metrics introduced here offer valuable benchmarks for assessing human-likeness in AI and highlight the importance of rigorous, quantitative imitation tests for AI development.
△ Less
Submitted 7 September, 2025; v1 submitted 23 November, 2022;
originally announced November 2022.
Look Twice: A Generalist Computational Model Predicts Return Fixations across Tasks and Species
Authors:
Mengmi Zhang,
Marcelo Armendariz,
Will Xiao,
Olivia Rose,
Katarina Bendtz,
Margaret Livingstone,
Carlos Ponce,
Gabriel Kreiman
Abstract:
Primates constantly explore their surroundings via saccadic eye movements that bring different parts of an image into high resolution. In addition to exploring new regions in the visual field, primates also make frequent return fixations, revisiting previously foveated locations. We systematically studied a total of 44,328 return fixations out of 217,440 fixations. Return fixations were ubiquitous…
▽ More
Primates constantly explore their surroundings via saccadic eye movements that bring different parts of an image into high resolution. In addition to exploring new regions in the visual field, primates also make frequent return fixations, revisiting previously foveated locations. We systematically studied a total of 44,328 return fixations out of 217,440 fixations. Return fixations were ubiquitous across different behavioral tasks, in monkeys and humans, both when subjects viewed static images and when subjects performed natural behaviors. Return fixations locations were consistent across subjects, tended to occur within short temporal offsets, and typically followed a 180-degree turn in saccadic direction. To understand the origin of return fixations, we propose a proof-of-principle, biologically-inspired and image-computable neural network model. The model combines five key modules: an image feature extractor, bottom-up saliency cues, task-relevant visual features, finite inhibition-of-return, and saccade size constraints. Even though there are no free parameters that are fine-tuned for each specific task, species, or condition, the model produces fixation sequences resembling the universal properties of return fixations. These results provide initial steps towards a mechanistic understanding of the trade-off between rapid foveal recognition and the need to scrutinize previous fixation locations.
△ Less
Submitted 14 October, 2022; v1 submitted 5 January, 2021;
originally announced January 2021.
