-
Evidence of Cognitive Deficits andDevelopmental Advances in Generative AI: A Clock Drawing Test Analysis
Authors:
Isaac R. Galatzer-Levy,
Jed McGiffin,
David Munday,
Xin Liu,
Danny Karmon,
Ilia Labzovsky,
Rivka Moroshko,
Amir Zait,
Daniel McDuff
Abstract:
Generative AI's rapid advancement sparks interest in its cognitive abilities, especially given its capacity for tasks like language understanding and code generation. This study explores how several recent GenAI models perform on the Clock Drawing Test (CDT), a neuropsychological assessment of visuospatial planning and organization. While models create clock-like drawings, they struggle with accur…
▽ More
Generative AI's rapid advancement sparks interest in its cognitive abilities, especially given its capacity for tasks like language understanding and code generation. This study explores how several recent GenAI models perform on the Clock Drawing Test (CDT), a neuropsychological assessment of visuospatial planning and organization. While models create clock-like drawings, they struggle with accurate time representation, showing deficits similar to mild-severe cognitive impairment (Wechsler, 2009). Errors include numerical sequencing issues, incorrect clock times, and irrelevant additions, despite accurate rendering of clock features. Only GPT 4 Turbo and Gemini Pro 1.5 produced the correct time, scoring like healthy individuals (4/4). A follow-up clock-reading test revealed only Sonnet 3.5 succeeded, suggesting drawing deficits stem from difficulty with numerical concepts. These findings may reflect weaknesses in visual-spatial understanding, working memory, or calculation, highlighting strengths in learned knowledge but weaknesses in reasoning. Comparing human and machine performance is crucial for understanding AI's cognitive capabilities and guiding development toward human-like cognitive functions.
△ Less
Submitted 15 October, 2024;
originally announced October 2024.
-
The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarks
Authors:
Isaac R. Galatzer-Levy,
David Munday,
Jed McGiffin,
Xin Liu,
Danny Karmon,
Ilia Labzovsky,
Rivka Moroshko,
Amir Zait,
Daniel McDuff
Abstract:
There is increasing interest in tracking the capabilities of general intelligence foundation models. This study benchmarks leading large language models and vision language models against human performance on the Wechsler Adult Intelligence Scale (WAIS-IV), a comprehensive, population-normed assessment of underlying human cognition and intellectual abilities, with a focus on the domains of VerbalC…
▽ More
There is increasing interest in tracking the capabilities of general intelligence foundation models. This study benchmarks leading large language models and vision language models against human performance on the Wechsler Adult Intelligence Scale (WAIS-IV), a comprehensive, population-normed assessment of underlying human cognition and intellectual abilities, with a focus on the domains of VerbalComprehension (VCI), Working Memory (WMI), and Perceptual Reasoning (PRI). Most models demonstrated exceptional capabilities in the storage, retrieval, and manipulation of tokens such as arbitrary sequences of letters and numbers, with performance on the Working Memory Index (WMI) greater or equal to the 99.5th percentile when compared to human population normative ability. Performance on the Verbal Comprehension Index (VCI) which measures retrieval of acquired information, and linguistic understanding about the meaning of words and their relationships to each other, also demonstrated consistent performance at or above the 98th percentile. Despite these broad strengths, we observed consistently poor performance on the Perceptual Reasoning Index (PRI; range 0.1-10th percentile) from multimodal models indicating profound inability to interpret and reason on visual information. Smaller and older model versions consistently performed worse, indicating that training data, parameter count and advances in tuning are resulting in significant advances in cognitive ability.
△ Less
Submitted 9 October, 2024;
originally announced October 2024.
-
First observation and study of the $K^{\pm} \rightarrow π^{0} π^{0} μ^{\pm} ν$ decay
Authors:
NA48/2 Collaboration,
:,
J. R. Batley,
G. Kalmus,
C. Lazzeroni,
D. J. Munday,
M. W. Slater,
S. A. Wotton,
R. Arcidiacono,
A. Ceccucci,
G. Bocquet,
N. Cabibbo,
D. Cundy,
V. Falaleev,
L. Gatignon,
M. Fidecaro,
A. Gonidec,
W. Kubischta,
A. Maier,
A. Norton,
M. Patel,
A. Peters,
S. Balev,
P. L. Frabetti,
E. Gersabeck
, et al. (100 additional authors not shown)
Abstract:
The NA48/2 experiment at CERN reports the first observation of the $K^{\pm} \rightarrow π^{0} π^{0} μ^{\pm} ν$ decay based on a sample of 2437 candidates with 15% background contamination collected in 2003--2004. The decay branching ratio in the kinematic region of the squared dilepton mass above $0.03$~GeV$^2/c^4$ is measured to be $(0.65 \pm 0.03) \times 10^{-6}$. The extrapolation to the full k…
▽ More
The NA48/2 experiment at CERN reports the first observation of the $K^{\pm} \rightarrow π^{0} π^{0} μ^{\pm} ν$ decay based on a sample of 2437 candidates with 15% background contamination collected in 2003--2004. The decay branching ratio in the kinematic region of the squared dilepton mass above $0.03$~GeV$^2/c^4$ is measured to be $(0.65 \pm 0.03) \times 10^{-6}$. The extrapolation to the full kinematic space, using a specific model, is found to be $(3.45 \pm 0.16) \times 10^{-6}$, in agreement with chiral perturbation theory predictions.
△ Less
Submitted 25 March, 2024; v1 submitted 31 October, 2023;
originally announced October 2023.
-
Search for $K^{+}\rightarrowπ^{+}ν\overlineν$ at NA62
Authors:
NA62 Collaboration,
G. Aglieri Rinella,
R. Aliberti,
F. Ambrosino,
R. Ammendola,
B. Angelucci,
A. Antonelli,
G. Anzivino,
R. Arcidiacono,
I. Azhinenko,
S. Balev,
M. Barbanera,
J. Bendotti,
A. Biagioni,
L. Bician,
C. Biino,
A. Bizzeti,
T. Blazek,
A. Blik,
B. Bloch-Devaux,
V. Bolotov,
V. Bonaiuto,
M. Boretto,
M. Bragadireanu,
D. Britton
, et al. (227 additional authors not shown)
Abstract:
$K^{+}\rightarrowπ^{+}ν\overlineν$ is one of the theoretically cleanest meson decay where to look for indirect effects of new physics complementary to LHC searches. The NA62 experiment at CERN SPS is designed to measure the branching ratio of this decay with 10\% precision. NA62 took data in pilot runs in 2014 and 2015 reaching the final designed beam intensity. The quality of 2015 data acquired,…
▽ More
$K^{+}\rightarrowπ^{+}ν\overlineν$ is one of the theoretically cleanest meson decay where to look for indirect effects of new physics complementary to LHC searches. The NA62 experiment at CERN SPS is designed to measure the branching ratio of this decay with 10\% precision. NA62 took data in pilot runs in 2014 and 2015 reaching the final designed beam intensity. The quality of 2015 data acquired, in view of the final measurement, will be presented.
△ Less
Submitted 24 July, 2018;
originally announced July 2018.
-
Implementation of a geometrically and energetically constrained mesoscale eddy parameterization in an ocean circulation model
Authors:
Julian Mak,
James R. Maddison,
David P. Marshall,
David R. Munday
Abstract:
The global stratification and circulation of the ocean and their sensitivities to changes in forcing depend crucially on the representation of the mesoscale eddy field. Here, a geometrically informed and energetically constrained parameterization framework for mesoscale eddies --- termed GEOMETRIC --- is proposed and implemented in three-dimensional primitive equation channel and sector models. Th…
▽ More
The global stratification and circulation of the ocean and their sensitivities to changes in forcing depend crucially on the representation of the mesoscale eddy field. Here, a geometrically informed and energetically constrained parameterization framework for mesoscale eddies --- termed GEOMETRIC --- is proposed and implemented in three-dimensional primitive equation channel and sector models. The GEOMETRIC framework closes mesoscale eddy fluxes according to the standard Gent--McWilliams scheme, but with the eddy transfer coefficient constrained by the depth-integrated eddy energy field, provided through a prognostic eddy energy budget evolving with the mean state. It is found that coarse resolution calculations employing GEOMETRIC broadly reproduce model sensitivities of the eddy permitting reference calculations in the emergent circumpolar transport, meridional overturning circulation profile and the depth-integrated eddy energy signature; in particular, eddy saturation emerges in the sector configuration. Some differences arise, attributed here to the simple prognostic eddy energy budget employed, to be improved upon in future investigations. The GEOMETRIC framework thus proposes a shift in paradigm, from a focus on how to close for eddy fluxes, to focusing on the representation of eddy energetics.
△ Less
Submitted 8 February, 2018;
originally announced February 2018.
-
ChPT tests at the NA48 and NA62 experiments at CERN
Authors:
NA48/2,
NA62 Collaborations,
:,
F. Ambrosino,
A. Antonelli,
G. Anzivino,
R. Arcidiacono,
W. Baldini,
S. Balev,
J. R. Batley,
M. Behler,
S. Bifani,
C. Biino,
A. Bizzeti,
B. Bloch-Devaux,
G. Bocquet,
V. Bolotov,
F. Bucci,
N. Cabibbo,
M. Calvetti,
N. Cartiglia,
A. Ceccucci,
P. Cenci,
C. Cerri,
C. Cheshkov
, et al. (137 additional authors not shown)
Abstract:
The NA48/2 Collaboration at CERN has accumulated unprecedented statistics of rare kaon decays in the Ke4 modes: Ke4(+-) ($K^\pm \to π^+ π^- e^\pm ν$) and Ke4(00) ($K^\pm \to π^0 π^0 e^\pm ν$) with nearly one percent background contamination. The detailed study of form factors and branching rates, based on these data, has been completed recently. The results brings new inputs to low energy strong i…
▽ More
The NA48/2 Collaboration at CERN has accumulated unprecedented statistics of rare kaon decays in the Ke4 modes: Ke4(+-) ($K^\pm \to π^+ π^- e^\pm ν$) and Ke4(00) ($K^\pm \to π^0 π^0 e^\pm ν$) with nearly one percent background contamination. The detailed study of form factors and branching rates, based on these data, has been completed recently. The results brings new inputs to low energy strong interactions description and tests of Chiral Perturbation Theory (ChPT) and lattice QCD calculations. In particular, new data support the ChPT prediction for a cusp in the $π^0π^0$ invariant mass spectrum at the two charged pions threshold for Ke4(00) decay. New final results from an analysis of about 400 $K^\pm \to π^\pm γγ$ rare decay candidates collected by the NA48/2 and NA62 experiments at CERN during low intensity runs with minimum bias trigger configurations are presented. The results include a model-independent decay rate measurement and fits to ChPT description.
△ Less
Submitted 29 January, 2016;
originally announced January 2016.
-
Prospects for $K^+ \to π^+ ν\bar{ ν}$ at CERN in NA62
Authors:
G. Aglieri Rinella,
R. Aliberti,
F. Ambrosino,
B. Angelucci,
A. Antonelli,
G. Anzivino,
R. Arcidiacono,
I. Azhinenko,
S. Balev,
J. Bendotti,
A. Biagioni,
C. Biino,
A. Bizzeti,
T. Blazek,
A. Blik,
B. Bloch-Devaux,
V. Bolotov,
V. Bonaiuto,
M. Bragadireanu,
D. Britton,
G. Britvich,
N. Brook,
F. Bucci,
V. Buescher,
F. Butin
, et al. (179 additional authors not shown)
Abstract:
The NA62 experiment will begin taking data in 2015. Its primary purpose is a 10% measurement of the branching ratio of the ultrarare kaon decay $K^+ \to π^+ ν\bar{ ν}$, using the decay in flight of kaons in an unseparated beam with momentum 75 GeV/c.The detector and analysis technique are described here.
The NA62 experiment will begin taking data in 2015. Its primary purpose is a 10% measurement of the branching ratio of the ultrarare kaon decay $K^+ \to π^+ ν\bar{ ν}$, using the decay in flight of kaons in an unseparated beam with momentum 75 GeV/c.The detector and analysis technique are described here.
△ Less
Submitted 1 November, 2014;
originally announced November 2014.
-
Recent NA48/2 and NA62 results
Authors:
F. Ambrosino,
A. Antonelli,
G. Anzivino,
R. Arcidiacono,
W. Baldini,
S. Balev,
J. R. Batley,
M. Behler,
S. Bifani,
C. Biino,
A. Bizzeti,
B. Bloch-Devaux,
G. Bocquet,
V. Bolotov,
F. Bucci,
N. Cabibbo,
M. Calvetti,
N. Cartiglia,
A. Ceccucci,
P. Cenci,
C. Cerri,
C. Cheshkov,
J. B. Cheze,
M. Clemencic,
G. Collazuol
, et al. (134 additional authors not shown)
Abstract:
The NA48/2 Collaboration at CERN has accumulated and analysed unprecedented statistics of rare kaon decays in the $K_{e4}$ modes: $K_{e4}(+-)$ ($K^\pm \to π^+ π^- e^\pm ν$) and $K_{e4}(00)$ ($K^\pm \to π^0 π^0 e^\pm ν$) with nearly one percent background contamination. It leads to the improved measurement of branching fractions and detailed form factor studies. New final results from the analysis…
▽ More
The NA48/2 Collaboration at CERN has accumulated and analysed unprecedented statistics of rare kaon decays in the $K_{e4}$ modes: $K_{e4}(+-)$ ($K^\pm \to π^+ π^- e^\pm ν$) and $K_{e4}(00)$ ($K^\pm \to π^0 π^0 e^\pm ν$) with nearly one percent background contamination. It leads to the improved measurement of branching fractions and detailed form factor studies. New final results from the analysis of 381 $K^\pm \to π^\pm γγ$ rare decay candidates collected by the NA48/2 and NA62 experiments at CERN are presented. The results include a decay rate measurement and fits to Chiral Perturbation Theory (ChPT) description.
△ Less
Submitted 4 August, 2014;
originally announced August 2014.
-
Measurement of the branching ratio of the decay $Ξ^{0}\rightarrow Σ^{+} μ^{-} \barν_μ$
Authors:
J. R. Batley,
G. E. Kalmus,
C. Lazzeroni,
D. J. Munday,
M. Patel,
M. W. Slater,
S. A. Wotton,
R. Arcidiacono,
G. Bocquet,
A. Ceccucci,
D. Cundy,
N. Doble,
V. Falaleev,
L. Gatignon,
A. Gonidec,
P. Grafstrom,
W. Kubischta,
F. Marchetto,
I. Mikulec,
A. Norton,
B. Panzer-Steindel,
P. Rubin,
H. Wahl,
E. Goudzovski,
P. Hristov
, et al. (88 additional authors not shown)
Abstract:
From the 2002 data taking with a neutral kaon beam extracted from the CERN-SPS, the NA48/1 experiment observed 97 $Ξ^{0}\rightarrow Σ^{+} μ^{-} \barν_μ$ candidates with a background contamination of $30.8 \pm 4.2$ events.
From this sample, the BR($Ξ^{0}\rightarrow Σ^{+} μ^{-} \barν_μ$) is measured to be $(2.17 \pm 0.32_{\mathrm{stat}}\pm 0.17_{\mathrm{syst}})\times10^{-6}$.
From the 2002 data taking with a neutral kaon beam extracted from the CERN-SPS, the NA48/1 experiment observed 97 $Ξ^{0}\rightarrow Σ^{+} μ^{-} \barν_μ$ candidates with a background contamination of $30.8 \pm 4.2$ events.
From this sample, the BR($Ξ^{0}\rightarrow Σ^{+} μ^{-} \barν_μ$) is measured to be $(2.17 \pm 0.32_{\mathrm{stat}}\pm 0.17_{\mathrm{syst}})\times10^{-6}$.
△ Less
Submitted 14 January, 2013; v1 submitted 13 December, 2012;
originally announced December 2012.
-
Empirical parameterization of the K+- -> pi+- pi0 pi0 decay Dalitz plot
Authors:
J. R. Batley,
A. J. Culling,
G. Kalmus,
C. Lazzeroni,
D. J. Munday,
M. W. Slater,
S. A. Wotton,
R. Arcidiacono,
G. Bocquet,
N. Cabibbo,
A. Ceccucci,
D. Cundy,
V. Falaleev,
M. Fidecaro,
L. Gatignon,
A. Gonidec,
W. Kubischta,
A. Norton,
A. Maier,
M. Patel,
A. Peters,
S. Balev,
P. L. Frabetti,
E. Goudzovski,
P. Hristov
, et al. (96 additional authors not shown)
Abstract:
As first observed by the NA48/2 experiment at the CERN SPS, the $\p0p0$ invariant mass (M00) distribution from $\kcnn$ decay shows a cusp-like anomaly at M00=2m+, where m+ is the charged pion mass. An analysis to extract the pi pi scattering lengths in the isospin I=0 and I=2 states, a0 and a2, respectively, has been recently reported. In the present work the Dalitz plot of this decay is fitted to…
▽ More
As first observed by the NA48/2 experiment at the CERN SPS, the $\p0p0$ invariant mass (M00) distribution from $\kcnn$ decay shows a cusp-like anomaly at M00=2m+, where m+ is the charged pion mass. An analysis to extract the pi pi scattering lengths in the isospin I=0 and I=2 states, a0 and a2, respectively, has been recently reported. In the present work the Dalitz plot of this decay is fitted to a new empirical parameterization suitable for practical purposes, such as Monte Carlo simulations of K+- -> pi+- pi0 pi0 decays.
△ Less
Submitted 7 April, 2010;
originally announced April 2010.
-
Observation of a cusp-like structure in the pizero-pizero invariant mass distribution from K+- ==> pi+- pizero pizero decay and determination of the pi-pi scattering lengths
Authors:
The NA48/2 Collaboration J. R. Batley,
C. Lazzeroni,
D. J. Munday,
M. W. Slater,
S. A. Wotton
Abstract:
We report the results from a study of ~23 Million K+- ==> pi+- pizero pizero decays recorded by the NA48/2 experiment at the CERN SPS, showing an anomaly in the pizero pizero invariant mass distribution in the region around 2m+, where m+ is the charged pion mass. This anomaly, never observed in previous experiments, can be interpreted as an effect due mainly to the final state charge exchange sc…
▽ More
We report the results from a study of ~23 Million K+- ==> pi+- pizero pizero decays recorded by the NA48/2 experiment at the CERN SPS, showing an anomaly in the pizero pizero invariant mass distribution in the region around 2m+, where m+ is the charged pion mass. This anomaly, never observed in previous experiments, can be interpreted as an effect due mainly to the final state charge exchange scattering process pi+ pi- ==> pizero pizero in K+- ==> pi+- pi+ pi- decay. It provides a precise determination of a0 - a2, the difference between the pi-pi scattering lengths in the isospin I=0 and I=2 states.
△ Less
Submitted 12 December, 2005; v1 submitted 28 November, 2005;
originally announced November 2005.
