-
Measuring Cognitive Abilities in the Wild: Validating a Population-Scale Game-Based Cognitive Assessment
Authors:
Mads Kock Pedersen,
Carlos Mauricio Castaño Díaz,
Qian Janice Wang,
Mario Alejandro Alba-Marrugo,
Ali Amidi,
Rajiv Vaid Basaiawmoit,
Carsten Bergenholtz,
Morten H. Christiansen,
Miroslav Gajdacz,
Ralph Hertwig,
Byurakn Ishkhanyan,
Kim Klyver,
Nicolai Ladegaard,
Kim Mathiasen,
Christine Parsons,
Janet Rafner,
Anders Ryom Villadsen,
Mikkel Wallentin,
Blanka Zana,
Jacob Friis Sherson
Abstract:
Rapid individual cognitive phenotyping holds the potential to revolutionize domains as wide-ranging as personalized learning, employment practices, and precision psychiatry. Going beyond limitations imposed by traditional lab-based experiments, new efforts have been underway towards greater ecological validity and participant diversity to capture the full range of individual differences in cogniti…
▽ More
Rapid individual cognitive phenotyping holds the potential to revolutionize domains as wide-ranging as personalized learning, employment practices, and precision psychiatry. Going beyond limitations imposed by traditional lab-based experiments, new efforts have been underway towards greater ecological validity and participant diversity to capture the full range of individual differences in cognitive abilities and behaviors across the general population. Building on this, we developed Skill Lab, a novel game-based tool that simultaneously assesses a broad suite of cognitive abilities while providing an engaging narrative. Skill Lab consists of six mini-games as well as 14 established cognitive ability tasks. Using a popular citizen science platform (N = 10725), we conducted a comprehensive validation in the wild of a game-based cognitive assessment suite. Based on the game and validation task data, we constructed reliable models to simultaneously predict eight cognitive abilities based on the users' in-game behavior. Follow-'-up validation tests revealed that the models can discriminate nuances contained within each separate cognitive ability as well as capture a shared main factor of generalized cognitive ability. Our game-based measures are five times faster to complete than the equivalent task-based measures and replicate previous findings on the decline of certain cognitive abilities with age in our large cross-sectional population sample (N = 6369). Taken together, our results demonstrate the feasibility of rapid in-the-wild systematic assessment of cognitive abilities as a promising first step towards population-scale benchmarking and individualized mental health diagnostics.
△ Less
Submitted 12 June, 2023; v1 submitted 11 September, 2020;
originally announced September 2020.
-
Human search in a fitness landscape: How to assess the difficulty of a search problem
Authors:
Oana Vuculescu,
Mads Kock Pedersen,
Jacob F. Sherson,
Carsten Bergenholtz
Abstract:
Computational modeling is widely used to study how humans and organizations search and solve problems in fields such as economics, management, cultural evolution, and computer science. We argue that current computational modeling research on human problem-solving needs to address several fundamental issues in order to generate more meaningful and falsifiable contributions. Based on comparative sim…
▽ More
Computational modeling is widely used to study how humans and organizations search and solve problems in fields such as economics, management, cultural evolution, and computer science. We argue that current computational modeling research on human problem-solving needs to address several fundamental issues in order to generate more meaningful and falsifiable contributions. Based on comparative simulations and a new type of visualization of how to assess the nature of the fitness landscape, we address two key assumptions that approaches such as the NK framework rely on: that the NK captures the continuum of the complexity of empirical fitness landscapes, and that search behavior is a distinct component, independent from the topology of the fitness landscape. We show the limitations of the most common approach to conceptualize how complex, or rugged, a landscape is, as well as how the nature of the fitness landscape is fundamentally intertwined with search behavior. Finally, we outline broader implications for how to simulate problem-solving.
△ Less
Submitted 21 June, 2020; v1 submitted 17 December, 2019;
originally announced December 2019.
-
Remote optimization of an ultra-cold atoms experiment by experts and citizen scientists
Authors:
Robert Heck,
Oana Vuculescu,
Jens Jakob Sørensen,
Jonathan Zoller,
Morten G. Andreasen,
Mark G. Bason,
Poul Ejlertsen,
Ottó Elíasson,
Pinja Haikka,
Jens S. Laustsen,
Lærke L. Nielsen,
Andrew Mao,
Romain Müller,
Mario Napolitano,
Mads K. Pedersen,
Aske R. Thorsen,
Carsten Bergenholtz,
Tommaso Calarco,
Simone Montangero,
Jacob F. Sherson
Abstract:
We introduce a novel remote interface to control and optimize the experimental production of Bose-Einstein condensates (BECs) and find improved solutions using two distinct implementations. First, a team of theoreticians employed a Remote version of their dCRAB optimization algorithm (RedCRAB), and second a gamified interface allowed 600 citizen scientists from around the world to participate in r…
▽ More
We introduce a novel remote interface to control and optimize the experimental production of Bose-Einstein condensates (BECs) and find improved solutions using two distinct implementations. First, a team of theoreticians employed a Remote version of their dCRAB optimization algorithm (RedCRAB), and second a gamified interface allowed 600 citizen scientists from around the world to participate in real-time optimization. Quantitative studies of player search behavior demonstrated that they collectively engage in a combination of local and global search. This form of adaptive search prevents premature convergence by the explorative behavior of low-performing players while high-performing players locally refine their solutions. In addition, many successful citizen science games have relied on a problem representation that directly engaged the visual or experiential intuition of the players. Here we demonstrate that citizen scientists can also be successful in an entirely abstract problem visualization. This gives encouragement that a much wider range of challenges could potentially be open to gamification in the future.
△ Less
Submitted 27 March, 2018; v1 submitted 7 September, 2017;
originally announced September 2017.
-
Leaderboard Effects on Player Performance in a Citizen Science Game
Authors:
Mads Kock Pedersen,
Nanna Ravn Rasmussen,
Jacob F. Sherson,
Rajiv Vaid Basaiawmoit
Abstract:
Quantum Moves is a citizen science game that investigates the ability of humans to solve complex physics challenges that are intractable for computers. During the launch of Quantum Moves in April 2016 the game's leaderboard function broke down resulting in a "no leaderboard" game experience for some players for a couple of days (though their scores were still displayed). The subsequent quick fix o…
▽ More
Quantum Moves is a citizen science game that investigates the ability of humans to solve complex physics challenges that are intractable for computers. During the launch of Quantum Moves in April 2016 the game's leaderboard function broke down resulting in a "no leaderboard" game experience for some players for a couple of days (though their scores were still displayed). The subsequent quick fix of an all-time Top 5 leaderboard, and the following long-term implementation of a personalized relative-position (infinite) leaderboard provided us with a unique opportunity to compare and investigate the effect of different leaderboard implementations on player performance in a points-driven citizen science game.
All three conditions were live sequentially during the game's initial influx of more than 150.000 players that stemmed from global press attention on Quantum Moves due the publication of a Nature paper about the use of Quantum Moves in solving a specific quantum physics problem. Thus, it has been possible to compare the three conditions and their influence on the performance (defined as a player's quality of game play related to a high-score) of over 4500 new players. These 4500 odd players in our three leaderboard-conditions have a similar demographic background based upon the time-window over which the implementations occurred and controlled against Player ID tags. Our results placed Condition 1 experience over condition 3 and in some cases even over condition 2 which goes against the general assumption that leaderboards enhance gameplay and its subsequent overuse as a an oft-relied upon element that designers slap onto a game to enhance said appeal. Our study thus questions the use of leaderboards as general performance enhancers in gamification contexts and brings some empirical rigor to an often under-reported but overused phenomenon.
△ Less
Submitted 12 July, 2017;
originally announced July 2017.
-
Knowledge Formation and Inter-Game Transfer With Classical and Quantum Physics
Authors:
Mads Kock Pedersen,
Camilla Clement Borre,
Andreas Lieberoth,
Jacob Sherson
Abstract:
In order to facilitate an intuitive understanding of classical physics concepts we have developed Potential Penguin - a game where players manipulate the landscape around a sliding penguin in order to control its movement. The learning goal of Potential Penguin is to familiarize players with kinetic energy and potential energy - the energies associated with movement and position in the landscape r…
▽ More
In order to facilitate an intuitive understanding of classical physics concepts we have developed Potential Penguin - a game where players manipulate the landscape around a sliding penguin in order to control its movement. The learning goal of Potential Penguin is to familiarize players with kinetic energy and potential energy - the energies associated with movement and position in the landscape respectively. The game levels introduce the concepts one by one, as players are tasked with sliding the penguin through a landscape towards a specific location, while keeping the velocity under control. When the player manipulates the landscape, the potential energy of the penguin is changed, which determines the penguin's movement. To build a strong connection between theory and game the analytical expressions for kinetic and potential energy are displayed during play with font sizes continually growing and shrinking according to changes in each energy type. With Potential Penguin we hope to study whether visualizing the amount of kinetic and potential energy through visible mathematical expressions generates a connection between the intuitive actions taken in the game and the underlying physics concepts. The knowledge about kinetic and potential energy gained with Potential Penguin can also be used to understand most of the physics behind the citizen science game Quantum Moves, which has the goal of building a working quantum computer. The two games share the principle of the core interaction - manipulating the potential-energy landscape. We aim to investigate whether a proficiency and understanding of Potential Penguin predicts a better performance in Quantum Moves and a deeper understanding of the quantum physics behind that game.
△ Less
Submitted 26 August, 2016;
originally announced August 2016.
-
DiffGame: Game-based mathematics learning for physics
Authors:
Mads Kock Pedersen,
Anette Svenningsen,
Niels Bonderup Dohn,
Andreas Lieberoth,
Jacob Sherson
Abstract:
Differentiation is a mathematical skill applied throughout science in order to describe the change of a function with respect to a dependent variable. Thus, an intuitive understanding of differentiation is necessary to work with the mathematical frameworks used to describe physical systems in the higher levels of education. In order to obtain this intuition repeated practice is required. This pape…
▽ More
Differentiation is a mathematical skill applied throughout science in order to describe the change of a function with respect to a dependent variable. Thus, an intuitive understanding of differentiation is necessary to work with the mathematical frameworks used to describe physical systems in the higher levels of education. In order to obtain this intuition repeated practice is required. This paper presents the development of DiffGame, which consists of a series of exercises that introduce the basic principles of differentiation for high-school students through game-like elements. DiffGame have been tested with 117 first-year students from a single Danish high school, who did not have any prior training in differentiation. The students' learning was assessed by the data obtained directly from DiffGame. The test demonstrated the efficacy of DiffGame, since students at all levels demonstrate a learning gain. In contrast to previous studies demonstrating most learning in the lower tier of students, the middle tier of students (based on overall performance) exhibits the largest learning gains.
△ Less
Submitted 25 May, 2016; v1 submitted 29 January, 2016;
originally announced January 2016.
-
Virtual learning environment for interactive engagement with advanced quantum mechanics
Authors:
Mads Kock Pedersen,
Birk Skyum,
Robert Heck,
Romain Müller,
Mark Bason,
Andreas Lieberoth,
Jacob F. Sherson
Abstract:
A virtual learning environment can engage university students in the learning process in ways that the traditional lectures and lab formats can not. We present our virtual learning environment \emph{StudentResearcher} which incorporates simulations, multiple-choice quizzes, video lectures and gamification into a learning path for quantum mechanics at the advanced university level. \emph{StudentRes…
▽ More
A virtual learning environment can engage university students in the learning process in ways that the traditional lectures and lab formats can not. We present our virtual learning environment \emph{StudentResearcher} which incorporates simulations, multiple-choice quizzes, video lectures and gamification into a learning path for quantum mechanics at the advanced university level. \emph{StudentResearcher} is built upon the experiences gathered from workshops with the citizen science game Quantum Moves at the high-school and university level, where the games were used extensively to illustrate the basic concepts of quantum mechanics. The first test of this new virtual learning environment was a 2014 course in advanced quantum mechanics at Aarhus University with 47 enrolled students. We found increased learning for the students who were more active on the platform independent of their previous performances.
△ Less
Submitted 23 March, 2016; v1 submitted 5 November, 2015;
originally announced November 2015.
-
Play or science?: a study of learning and framing in crowdscience games
Authors:
Andreas Lieberoth,
Mads Kock Pedersen,
Jacob Sherson
Abstract:
Crowdscience games may hold unique potentials as learning opportunities compared to games made for fun or education. They are part of an actual science problem solving process: By playing, players help scientists, and thereby interact with real continuous research processes. This mixes the two worlds of play and science in new ways. During usability testing we discovered that users of the crowdsci…
▽ More
Crowdscience games may hold unique potentials as learning opportunities compared to games made for fun or education. They are part of an actual science problem solving process: By playing, players help scientists, and thereby interact with real continuous research processes. This mixes the two worlds of play and science in new ways. During usability testing we discovered that users of the crowdscience game Quantum Dreams tended to answer questions in game terms, even when directed explicitly to give science explanations.We then examined these competing frames of understanding through a mixed correlational and grounded theory analysis. This essay presents the core ideas of crowdscience games as learning opportunities, and reports how a group of players used "game", "science" and "conceptual" frames to interpret their experience. Our results suggest that oscillating between the frames instead of sticking to just one led to the largest number of correct science interpretations, as players could participate legitimately and autonomously at multiple levels of understanding.
△ Less
Submitted 23 October, 2015;
originally announced October 2015.
-
Exploring the Quantum Speed Limit with Computer Games
Authors:
Jens Jakob W. H. Sørensen,
Mads Kock Pedersen,
Michael Munch,
Pinja Haikka,
Jesper Halkjær Jensen,
Tilo Planke,
Morten Ginnerup Andreasen,
Miroslav Gajdacz,
Klaus Mølmer,
Andreas Lieberoth,
Jacob F. Sherson,
Quantum Moves players
Abstract:
Humans routinely solve problems of immense computational complexity by intuitively forming simple, low-dimensional heuristic strategies. Citizen science exploits this ability by presenting scientific research problems to non-experts. Gamification is an effective tool for attracting citizen scientists to provide solutions to research problems. While citizen science games Foldit, EteRNA and EyeWire…
▽ More
Humans routinely solve problems of immense computational complexity by intuitively forming simple, low-dimensional heuristic strategies. Citizen science exploits this ability by presenting scientific research problems to non-experts. Gamification is an effective tool for attracting citizen scientists to provide solutions to research problems. While citizen science games Foldit, EteRNA and EyeWire have been used successfully to study protein and RNA folding and neuron mapping, so far gamification has not been applied to problems in quantum physics. Does the fact that everyday experiences are based on classical physics hinder the use of non-expert citizen scientists in the realm of quantum mechanics? Here we report on Quantum Moves, an online platform gamifying optimization problems in quantum physics. We show that human players are able to find solutions to difficult problems associated with the task of quantum computing. Players succeed where purely numerical optimization fails, and analyses of their solutions provide insights into the problem of optimization of a more profound and general nature. Based on player strategies, we have thus developed a new, few-parameter heuristic optimization method which efficiently outperforms the most prominent established numerical methods. The numerical complexity associated with time-optimal solutions increases for shorter process durations. To better understand this, we have made a low-dimensional rendering of the optimization landscape. These studies show why traditional optimization methods fail near the quantum speed limit, and they bring promise that combined analyses of optimization landscapes and heuristic solution strategies may benefit wider classes of optimization problems in quantum physics and beyond.
△ Less
Submitted 11 September, 2016; v1 submitted 26 June, 2015;
originally announced June 2015.
-
Getting Humans to do Quantum Optimization - User Acquisition, Engagement and Early Results from the Citizen Cyberscience Game Quantum Moves
Authors:
Andreas Lieberoth,
Mads Kock Pedersen,
Andreea Catalina Marin,
Tilo Planke,
Jacob Friis Sherson
Abstract:
The game Quantum Moves was designed to pit human players against computer algorithms, combining their solutions into hybrid optimization to control a scalable quantum computer. In this midstream report, we open our design process and describe the series of constitutive building stages going into a quantum physics citizen science game. We present our approach from designing a core gameplay around q…
▽ More
The game Quantum Moves was designed to pit human players against computer algorithms, combining their solutions into hybrid optimization to control a scalable quantum computer. In this midstream report, we open our design process and describe the series of constitutive building stages going into a quantum physics citizen science game. We present our approach from designing a core gameplay around quantum simulations, to putting extra game elements in place in order to frame, structure, and motivate players' difficult path from curious visitors to competent science contributors. The player base is extremely diverse - for instance, two top players are a 40 year old female accountant and a male taxi driver. Among statistical predictors for retention and in-game high scores, the data from our first year suggest that people recruited based on real-world physics interest and via real-world events, but only with an intermediate science education, are more likely to become engaged and skilled contributors. Interestingly, female players tended to perform better than male players, even though men played more games per day. To understand this relationship, we explore the profiles of our top players in more depth. We discuss in-world and in-game performance factors departing in psychological theories of intrinsic and extrinsic motivation, and the implications for using real live humans to do hybrid optimization via initially simple, but ultimately very cognitively complex games.
△ Less
Submitted 26 June, 2015;
originally announced June 2015.
-
Gamification of Quantum Mechanics Teaching
Authors:
Ole Eggers Bjælde,
Mads Kock Pedersen,
Jacob Sherson
Abstract:
In this small scale study we demonstrate how a gamified teaching setup can be used effectively to support student learning in a quantum mechanics course. The quantum mechanics games were research games, which were played during lectures and the learning was measured with a pretest/posttest method with promising results. The study works as a pilot study to guide the planning of quantum mechanics co…
▽ More
In this small scale study we demonstrate how a gamified teaching setup can be used effectively to support student learning in a quantum mechanics course. The quantum mechanics games were research games, which were played during lectures and the learning was measured with a pretest/posttest method with promising results. The study works as a pilot study to guide the planning of quantum mechanics courses in the future at Aarhus University in Denmark.
△ Less
Submitted 26 June, 2015;
originally announced June 2015.
-
Many-body state engineering using measurements and fixed unitary dynamics
Authors:
Mads Kock Pedersen,
Jens Jakob W. H. Sørensen,
Malte C. Tichy,
Jacob F. Sherson
Abstract:
We develop a scheme to prepare a desired state or subspace in high-dimensional Hilbert-spaces using repeated applications of a single static projection operator onto the desired target and fixed unitary dynamics. Benchmarks against other control schemes, performed on generic Hamiltonians and on Bose-Hubbard systems, establish the competitiveness of the method. As a concrete application of the cont…
▽ More
We develop a scheme to prepare a desired state or subspace in high-dimensional Hilbert-spaces using repeated applications of a single static projection operator onto the desired target and fixed unitary dynamics. Benchmarks against other control schemes, performed on generic Hamiltonians and on Bose-Hubbard systems, establish the competitiveness of the method. As a concrete application of the control of mesoscopic atomic samples in optical lattices we demonstrate the near deterministic preparation of Schrödinger cat states of all atoms residing on either the odd or the even sites.
△ Less
Submitted 20 November, 2014; v1 submitted 3 June, 2014;
originally announced June 2014.
-
Non-adiabatic many-atom quantum state control in few-well systems
Authors:
Malte C. Tichy,
Mads Kock Pedersen,
Klaus Mølmer,
Jacob F. Sherson
Abstract:
We present a fast scheme for arbitrary unitary control of interacting bosonic atoms in a double-well. Assuming fixed inter-well tunnelling rate and intra-well interaction strength, we control the many-atom state by a discrete sequence of shifts of the single-well energies. For strong interactions, resonant tunnelling transitions implement beam-splitter U(2) rotations among atom number eigenstates,…
▽ More
We present a fast scheme for arbitrary unitary control of interacting bosonic atoms in a double-well. Assuming fixed inter-well tunnelling rate and intra-well interaction strength, we control the many-atom state by a discrete sequence of shifts of the single-well energies. For strong interactions, resonant tunnelling transitions implement beam-splitter U(2) rotations among atom number eigenstates, which can be combined and, thus, permit full controllability. By numerically optimizing such sequences of couplings at avoided level crossings (CALC), we extend the realm of full controllability to a wide range of realistic interaction parameters, while we remain in the simple control space. We demonstrate the efficiency and the high achievable fidelity of our proposal with non-adiabatic population transfer, N00N-state creation, a C-NOT gate, and a transistor-like, conditional evolution of several atoms.
△ Less
Submitted 13 May, 2013; v1 submitted 14 January, 2013;
originally announced January 2013.
