-
Correspondence of high-dimensional emotion structures elicited by video clips between humans and Multimodal LLMs
Authors:
Haruka Asanuma,
Naoko Koide-Majima,
Ken Nakamura,
Takato Horii,
Shinji Nishimoto,
Masafumi Oizumi
Abstract:
Recent studies have revealed that human emotions exhibit a high-dimensional, complex structure. A full capturing of this complexity requires new approaches, as conventional models that disregard high dimensionality risk overlooking key nuances of human emotions. Here, we examined the extent to which the latest generation of rapidly evolving Multimodal Large Language Models (MLLMs) capture these hi…
▽ More
Recent studies have revealed that human emotions exhibit a high-dimensional, complex structure. A full capturing of this complexity requires new approaches, as conventional models that disregard high dimensionality risk overlooking key nuances of human emotions. Here, we examined the extent to which the latest generation of rapidly evolving Multimodal Large Language Models (MLLMs) capture these high-dimensional, intricate emotion structures, including capabilities and limitations. Specifically, we compared self-reported emotion ratings from participants watching videos with model-generated estimates (e.g., Gemini or GPT). We evaluated performance not only at the individual video level but also from emotion structures that account for inter-video relationships. At the level of simple correlation between emotion structures, our results demonstrated strong similarity between human and model-inferred emotion structures. To further explore whether the similarity between humans and models is at the signle item level or the coarse-categorical level, we applied Gromov Wasserstein Optimal Transport. We found that although performance was not necessarily high at the strict, single-item level, performance across video categories that elicit similar emotions was substantial, indicating that the model could infer human emotional experiences at the category level. Our results suggest that current state-of-the-art MLLMs broadly capture the complex high-dimensional emotion structures at the category level, as well as their apparent limitations in accurately capturing entire structures at the single-item level.
△ Less
Submitted 23 May, 2025; v1 submitted 19 May, 2025;
originally announced May 2025.
-
Statistical Mechanical Analysis of Catastrophic Forgetting in Continual Learning with Teacher and Student Networks
Authors:
Haruka Asanuma,
Shiro Takagi,
Yoshihiro Nagano,
Yuki Yoshida,
Yasuhiko Igarashi,
Masato Okada
Abstract:
When a computational system continuously learns from an ever-changing environment, it rapidly forgets its past experiences. This phenomenon is called catastrophic forgetting. While a line of studies has been proposed with respect to avoiding catastrophic forgetting, most of the methods are based on intuitive insights into the phenomenon, and their performances have been evaluated by numerical expe…
▽ More
When a computational system continuously learns from an ever-changing environment, it rapidly forgets its past experiences. This phenomenon is called catastrophic forgetting. While a line of studies has been proposed with respect to avoiding catastrophic forgetting, most of the methods are based on intuitive insights into the phenomenon, and their performances have been evaluated by numerical experiments using benchmark datasets. Therefore, in this study, we provide the theoretical framework for analyzing catastrophic forgetting by using teacher-student learning. Teacher-student learning is a framework in which we introduce two neural networks: one neural network is a target function in supervised learning, and the other is a learning neural network. To analyze continual learning in the teacher-student framework, we introduce the similarity of the input distribution and the input-output relationship of the target functions as the similarity of tasks. In this theoretical framework, we also provide a qualitative understanding of how a single-layer linear learning neural network forgets tasks. Based on the analysis, we find that the network can avoid catastrophic forgetting when the similarity among input distributions is small and that of the input-output relationship of the target functions is large. The analysis also suggests that a system often exhibits a characteristic phenomenon called overshoot, which means that even if the learning network has once undergone catastrophic forgetting, it is possible that the network may perform reasonably well after further learning of the current task.
△ Less
Submitted 16 May, 2021;
originally announced May 2021.
-
Injection induced seismicity size distribution dependent on shear stress
Authors:
Yusuke Mukuhira,
Michael C. Fehler,
Takatoshi Ito,
Hiroshi Asanuma,
Markus O. Häring
Abstract:
Like natural seismicity, induced seismicity caused by fluid injection also shows a power law size distribution, and its gradient b-value (ratio of small to large earthquakes) is often used for seismic hazard analysis. Despite well-known relationship that b-value is negatively correlated with differential stress for natural earthquakes, there is no understanding of the physical causes for b-value v…
▽ More
Like natural seismicity, induced seismicity caused by fluid injection also shows a power law size distribution, and its gradient b-value (ratio of small to large earthquakes) is often used for seismic hazard analysis. Despite well-known relationship that b-value is negatively correlated with differential stress for natural earthquakes, there is no understanding of the physical causes for b-value variations in injection-induced seismicity in the scale where the differential is nearly constant. We investigate a b-value dependence on the relative shear stress acting on existing fractures and show that the seismicity occurring along existing fractures with high shear stress have significantly lower b-values than does that associated with lower shear stress fractures. The b-value for injection induced seismicity is dependent on relative shear stress on faults. Our results provide a novel physical explanation for the b-value variations of induced seismicity.
△ Less
Submitted 30 May, 2020;
originally announced June 2020.
