EAZY: Eliminating Hallucinations in LVLMs by Zeroing out Hallucinatory Image Tokens
Authors:
Liwei Che,
Tony Qingze Liu,
Jing Jia,
Weiyi Qin,
Ruixiang Tang,
Vladimir Pavlovic
Abstract:
Despite their remarkable potential, Large Vision-Language Models (LVLMs) still face challenges with object hallucination, a problem where their generated outputs mistakenly incorporate objects that do not actually exist. Although most works focus on addressing this issue within the language-model backbone, our work shifts the focus to the image input source, investigating how specific image tokens…
▽ More
Despite their remarkable potential, Large Vision-Language Models (LVLMs) still face challenges with object hallucination, a problem where their generated outputs mistakenly incorporate objects that do not actually exist. Although most works focus on addressing this issue within the language-model backbone, our work shifts the focus to the image input source, investigating how specific image tokens contribute to hallucinations. Our analysis reveals a striking finding: a small subset of image tokens with high attention scores are the primary drivers of object hallucination. By removing these hallucinatory image tokens (only 1.5% of all image tokens), the issue can be effectively mitigated. This finding holds consistently across different models and datasets. Building on this insight, we introduce EAZY, a novel, training-free method that automatically identifies and Eliminates hAllucinations by Zeroing out hallucinatorY image tokens. We utilize EAZY for unsupervised object hallucination detection, achieving 15% improvement compared to previous methods. Additionally, EAZY demonstrates remarkable effectiveness in mitigating hallucinations while preserving model utility and seamlessly adapting to various LVLM architectures.
△ Less
Submitted 10 March, 2025;
originally announced March 2025.
Multiple-models prediction for light neutron-rich isotopes cross section by $Q_g$ systematics in $^{40}$Ar projectile fragmentation reactions
Authors:
X. B. Wei,
H. L. Wei,
C. W. Ma,
C. Y. Qiao,
Y. F. Guo,
J. Pu,
K. X. Cheng,
Y. T. Wang,
Z. X. Wang,
T. R. Zhou,
D. Peng,
S. T. Wang,
S. W. Tang,
Y. H. Yu,
X. H. Zhang,
Y. Z. Sun,
S. Y. Jin,
G. L. Zhang,
X. Jiang,
Z. Y. Li,
Y. F. Xu,
F. H. Lu,
T. Q. Liu
Abstract:
Precise predictions for nuclei near drip lines are crucial for experiments in new generation of rare isotope facilities. A multi-models investigation of the $Q_g$ systematics for fragments production cross sections, with $Q_g$ defined as the difference of mass excess (ME) between the projectile ($Z_{p}, A_{p}$) and the fragment ($Z_{f}, A_{f}$) nuclei $Q_{g}=ME(Z_{p}, A_{p})-ME(Z_{f}, A_{f})$, has…
▽ More
Precise predictions for nuclei near drip lines are crucial for experiments in new generation of rare isotope facilities. A multi-models investigation of the $Q_g$ systematics for fragments production cross sections, with $Q_g$ defined as the difference of mass excess (ME) between the projectile ($Z_{p}, A_{p}$) and the fragment ($Z_{f}, A_{f}$) nuclei $Q_{g}=ME(Z_{p}, A_{p})-ME(Z_{f}, A_{f})$, has been performed to verify the model prediction abilities for light neutron-rich isotopes in measured $^{40}$Ar + $^9$Be projectile fragmentation reactions from 57$A$ MeV to 1$A$ GeV. The models used are the FRACS parametrizations and the newly developed Bayesian neural networks (BNN) model. %method The results show that FRACS, BNN, and $Q_g$ extrapolations are generally consistent, except for fragments near the nuclear mass of the projectile. Additionally, both measured data and model extrapolations provide evidence for a shell closure at $N=$ 16 in fluorine and neon, as well as the disappearance of the traditional magic number $N=$ 20 in neon, sodium and magnesium.
△ Less
Submitted 14 September, 2024;
originally announced September 2024.
