Controversial stimuli: pitting neural networks against each other as models of human recognition
Authors:
Tal Golan,
Prashant C. Raju,
Nikolaus Kriegeskorte
Abstract:
Distinct scientific theories can make similar predictions. To adjudicate between theories, we must design experiments for which the theories make distinct predictions. Here we consider the problem of comparing deep neural networks as models of human visual recognition. To efficiently compare models' ability to predict human responses, we synthesize controversial stimuli: images for which different…
▽ More
Distinct scientific theories can make similar predictions. To adjudicate between theories, we must design experiments for which the theories make distinct predictions. Here we consider the problem of comparing deep neural networks as models of human visual recognition. To efficiently compare models' ability to predict human responses, we synthesize controversial stimuli: images for which different models produce distinct responses. We applied this approach to two visual recognition tasks, handwritten digits (MNIST) and objects in small natural images (CIFAR-10). For each task, we synthesized controversial stimuli to maximize the disagreement among models which employed different architectures and recognition algorithms. Human subjects viewed hundreds of these stimuli, as well as natural examples, and judged the probability of presence of each digit/object category in each image. We quantified how accurately each model predicted the human judgments. The best performing models were a generative Analysis-by-Synthesis model (based on variational autoencoders) for MNIST and a hybrid discriminative-generative Joint Energy Model for CIFAR-10. These DNNs, which model the distribution of images, performed better than purely discriminative DNNs, which learn only to map images to labels. None of the candidate models fully explained the human responses. Controversial stimuli generalize the concept of adversarial examples, obviating the need to assume a ground-truth model. Unlike natural images, controversial stimuli are not constrained to the stimulus distribution models are trained on, thus providing severe out-of-distribution tests that reveal the models' inductive biases. Controversial stimuli therefore provide powerful probes of discrepancies between models and human perception.
△ Less
Submitted 15 July, 2020; v1 submitted 20 November, 2019;
originally announced November 2019.
A Theory on Formatting Sensory Input for Cognition
Authors:
Prashant C. Raju
Abstract:
Over the last few decades, a lot of progress has been made in understanding different aspects of the brain's ability to form abstract representations, but a specific mechanism for how they are created and used remains to emerge. Here, we review recent findings on the subject and we propose a mechanism for the dynamics of forming abstract representations, where the formation of local connectivity i…
▽ More
Over the last few decades, a lot of progress has been made in understanding different aspects of the brain's ability to form abstract representations, but a specific mechanism for how they are created and used remains to emerge. Here, we review recent findings on the subject and we propose a mechanism for the dynamics of forming abstract representations, where the formation of local connectivity in neural networks determines the of search terms between the prefrontal cortex and the hippocampus, as well as the amount of detail that is transcribed into abstract representations.
△ Less
Submitted 27 February, 2020; v1 submitted 8 September, 2019;
originally announced September 2019.
