-
From Model to Classroom: Evaluating Generated MCQs for Portuguese with Narrative and Difficulty Concerns
Authors:
Bernardo Leite,
Henrique Lopes Cardoso,
Pedro Pinto,
Abel Ferreira,
Luís Abreu,
Isabel Rangel,
Sandra Monteiro
Abstract:
While MCQs are valuable for learning and evaluation, manually creating them with varying difficulty levels and targeted reading skills remains a time-consuming and costly task. Recent advances in generative AI provide an opportunity to automate MCQ generation efficiently. However, assessing the actual quality and reliability of generated MCQs has received limited attention -- particularly regardin…
▽ More
While MCQs are valuable for learning and evaluation, manually creating them with varying difficulty levels and targeted reading skills remains a time-consuming and costly task. Recent advances in generative AI provide an opportunity to automate MCQ generation efficiently. However, assessing the actual quality and reliability of generated MCQs has received limited attention -- particularly regarding cases where generation fails. This aspect becomes particularly important when the generated MCQs are meant to be applied in real-world settings. Additionally, most MCQ generation studies focus on English, leaving other languages underexplored. This paper investigates the capabilities of current generative models in producing MCQs for reading comprehension in Portuguese, a morphologically rich language. Our study focuses on generating MCQs that align with curriculum-relevant narrative elements and span different difficulty levels. We evaluate these MCQs through expert review and by analyzing the psychometric properties extracted from student responses to assess their suitability for elementary school students. Our results show that current models can generate MCQs of comparable quality to human-authored ones. However, we identify issues related to semantic clarity and answerability. Also, challenges remain in generating distractors that engage students and meet established criteria for high-quality MCQ option design.
△ Less
Submitted 18 June, 2025;
originally announced June 2025.
-
DSSE: a drone swarm search environment
Authors:
Manuel Castanares,
Luis F. S. Carrete,
Enrico F. Damiani,
Leonardo D. M. de Abreu,
José Fernando B. Brancalion,
Fabrício J. Barth
Abstract:
The Drone Swarm Search project is an environment, based on PettingZoo, that is to be used in conjunction with multi-agent (or single-agent) reinforcement learning algorithms. It is an environment in which the agents (drones), have to find the targets (shipwrecked people). The agents do not know the position of the target and do not receive rewards related to their own distance to the target(s). Ho…
▽ More
The Drone Swarm Search project is an environment, based on PettingZoo, that is to be used in conjunction with multi-agent (or single-agent) reinforcement learning algorithms. It is an environment in which the agents (drones), have to find the targets (shipwrecked people). The agents do not know the position of the target and do not receive rewards related to their own distance to the target(s). However, the agents receive the probabilities of the target(s) being in a certain cell of the map. The aim of this project is to aid in the study of reinforcement learning algorithms that require dynamic probabilities as inputs.
△ Less
Submitted 12 July, 2023;
originally announced July 2023.
-
MSE estimates for multitaper spectral estimation and off-grid compressive sensing
Authors:
Luís Daniel Abreu,
José Luis Romero
Abstract:
We obtain estimates for the Mean Squared Error (MSE) for the multitaper spectral estimator and certain compressive acquisition methods for multi-band signals. We confirm a fact discovered by Thomson [Spectrum estimation and harmonic analysis, Proc. IEEE, 1982]: assuming bandwidth $W$ and $N$ time domain observations, the average of the square of the first $K=2NW$ Slepian functions approaches, as…
▽ More
We obtain estimates for the Mean Squared Error (MSE) for the multitaper spectral estimator and certain compressive acquisition methods for multi-band signals. We confirm a fact discovered by Thomson [Spectrum estimation and harmonic analysis, Proc. IEEE, 1982]: assuming bandwidth $W$ and $N$ time domain observations, the average of the square of the first $K=2NW$ Slepian functions approaches, as $K$ grows, an ideal band-pass kernel for the interval $[-W,W]$. We provide an analytic proof of this fact and measure the corresponding rate of convergence in the $L^{1}$ norm. This validates a heuristic approximation used to control the MSE of the multitaper estimator. The estimates have also consequences for the method of compressive acquisition of multi-band signals introduced by Davenport and Wakin, giving MSE approximation bounds for the dictionary formed by modulation of the critical number of prolates.
△ Less
Submitted 20 June, 2017; v1 submitted 23 March, 2017;
originally announced March 2017.
-
The average singular value of a complex random matrix decreases with dimension
Authors:
Luís Daniel Abreu
Abstract:
We obtain a recurrence relation in $d$ for the average singular value $% α(d)$ of a complex valued $d\times d$\ matrix $\frac{1}{\sqrt{d}}X$ with random i.i.d., N( 0,1) entries, and use it to show that $α(d)$ decreases monotonically with $d$ to the limit given by the Marchenko-Pastur distribution.\ The monotonicity of $α(d)$ has been recently conjectured by Bandeira, Kennedy and Singer in their st…
▽ More
We obtain a recurrence relation in $d$ for the average singular value $% α(d)$ of a complex valued $d\times d$\ matrix $\frac{1}{\sqrt{d}}X$ with random i.i.d., N( 0,1) entries, and use it to show that $α(d)$ decreases monotonically with $d$ to the limit given by the Marchenko-Pastur distribution.\ The monotonicity of $α(d)$ has been recently conjectured by Bandeira, Kennedy and Singer in their study of the Little Grothendieck problem over the unitary group $\mathcal{U}_{d}$ \cite{BKS}, a combinatorial optimization problem. The result implies sharp global estimates for $α(d)$, new bounds for the expected minimum and maximum singular values, and a lower bound for the ratio of the expected maximum and the expected minimum singular value. The proof is based on a connection with the theory of Turán determinants of orthogonal polynomials. We also discuss some applications to the problem that originally motivated the conjecture.
△ Less
Submitted 6 March, 2023; v1 submitted 1 June, 2016;
originally announced June 2016.
-
Super-wavelets versus poly-Bergman spaces
Authors:
Luis Daniel Abreu
Abstract:
Motivated by potential applications in multiplexing and by recent results on Gabor analysis with Hermite windows due to Gröchenig and Lyubarskii, we investigate vector-valued wavelet transforms and vector-valued wavelet frames, which constitute special cases of super-wavelets, with a particular attention to the case when the analyzing wavelet vector is related to Fourier transforms of Laguerre f…
▽ More
Motivated by potential applications in multiplexing and by recent results on Gabor analysis with Hermite windows due to Gröchenig and Lyubarskii, we investigate vector-valued wavelet transforms and vector-valued wavelet frames, which constitute special cases of super-wavelets, with a particular attention to the case when the analyzing wavelet vector is related to Fourier transforms of Laguerre functions. We construct an isometric isomorphism between $L^{2}(\mathbb{R}^{+},\mathbf{C}^{n})$ and poly-Bergman spaces, with a view to relate the sampling sequences in the poly-Bergman spaces to the wavelet frames and super-frames with the windows $Φ_{n}$. One of the applications of the theory is a proof that $b\ln a<2π(n+1)$ is a necessary condition for the (scalar) wavelet frame associated to the $Φ_{n}$ to exist. This seems to be the first known result of this type outside the setting of analytic functions (the case $n=0$, which has been completely studied by Seip in 1993).
△ Less
Submitted 25 September, 2009;
originally announced September 2009.
-
Bilinear biorthogonal expansions and the Dunkl kernel on the real line
Authors:
L. D. Abreu,
Ó. Ciaurri,
J. L. Varona
Abstract:
We study an extension of the classical Paley-Wiener space structure, which is based on bilinear expansions of integral kernels into biorthogonal sequences of functions. The structure includes both sampling expansions and Fourier-Neumann type series as special cases, and it also provides a bilinear expansion for the Dunkl kernel (in the rank 1 case) which is a Dunkl analogue of Gegenbauer's expansi…
▽ More
We study an extension of the classical Paley-Wiener space structure, which is based on bilinear expansions of integral kernels into biorthogonal sequences of functions. The structure includes both sampling expansions and Fourier-Neumann type series as special cases, and it also provides a bilinear expansion for the Dunkl kernel (in the rank 1 case) which is a Dunkl analogue of Gegenbauer's expansion of the plane wave and the corresponding sampling expansions. In fact, we show how to derive sampling and Fourier-Neumann type expansions from the results related to the bilinear expansion for the Dunkl kernel.
△ Less
Submitted 5 November, 2012; v1 submitted 31 August, 2009;
originally announced September 2009.
