-
Measuring How LLMs Internalize Human Psychological Concepts: A preliminary analysis
Authors:
Hiro Taiyo Hamada,
Ippei Fujisawa,
Genji Kawakita,
Yuki Yamada
Abstract:
Large Language Models (LLMs) such as ChatGPT have shown remarkable abilities in producing human-like text. However, it is unclear how accurately these models internalize concepts that shape human thought and behavior. Here, we developed a quantitative framework to assess concept alignment between LLMs and human psychological dimensions using 43 standardized psychological questionnaires, selected f…
▽ More
Large Language Models (LLMs) such as ChatGPT have shown remarkable abilities in producing human-like text. However, it is unclear how accurately these models internalize concepts that shape human thought and behavior. Here, we developed a quantitative framework to assess concept alignment between LLMs and human psychological dimensions using 43 standardized psychological questionnaires, selected for their established validity in measuring distinct psychological constructs. Our method evaluates how accurately language models reconstruct and classify questionnaire items through pairwise similarity analysis. We compared resulting cluster structures with the original categorical labels using hierarchical clustering. A GPT-4 model achieved superior classification accuracy (66.2\%), significantly outperforming GPT-3.5 (55.9\%) and BERT (48.1\%), all exceeding random baseline performance (31.9\%). We also demonstrated that the estimated semantic similarity from GPT-4 is associated with Pearson's correlation coefficients of human responses in multiple psychological questionnaires. This framework provides a novel approach to evaluate the alignment of the human-LLM concept and identify potential representational biases. Our findings demonstrate that modern LLMs can approximate human psychological constructs with measurable accuracy, offering insights for developing more interpretable AI systems.
△ Less
Submitted 28 June, 2025;
originally announced June 2025.
-
Automatic hip osteoarthritis grading with uncertainty estimation from computed tomography using digitally-reconstructed radiographs
Authors:
Masachika Masuda,
Mazen Soufi,
Yoshito Otake,
Keisuke Uemura,
Sotaro Kono,
Kazuma Takashima,
Hidetoshi Hamada,
Yi Gu,
Masaki Takao,
Seiji Okada,
Nobuhiko Sugano,
Yoshinobu Sato
Abstract:
Progression of hip osteoarthritis (hip OA) leads to pain and disability, likely leading to surgical treatment such as hip arthroplasty at the terminal stage. The severity of hip OA is often classified using the Crowe and Kellgren-Lawrence (KL) classifications. However, as the classification is subjective, we aimed to develop an automated approach to classify the disease severity based on the two g…
▽ More
Progression of hip osteoarthritis (hip OA) leads to pain and disability, likely leading to surgical treatment such as hip arthroplasty at the terminal stage. The severity of hip OA is often classified using the Crowe and Kellgren-Lawrence (KL) classifications. However, as the classification is subjective, we aimed to develop an automated approach to classify the disease severity based on the two grades using digitally-reconstructed radiographs (DRRs) from CT images. Automatic grading of the hip OA severity was performed using deep learning-based models. The models were trained to predict the disease grade using two grading schemes, i.e., predicting the Crowe and KL grades separately, and predicting a new ordinal label combining both grades and representing the disease progression of hip OA. The models were trained in classification and regression settings. In addition, the model uncertainty was estimated and validated as a predictor of classification accuracy. The models were trained and validated on a database of 197 hip OA patients, and externally validated on 52 patients. The model accuracy was evaluated using exact class accuracy (ECA), one-neighbor class accuracy (ONCA), and balanced accuracy.The deep learning models produced a comparable accuracy of approximately 0.65 (ECA) and 0.95 (ONCA) in the classification and regression settings. The model uncertainty was significantly larger in cases with large classification errors (P<6e-3). In this study, an automatic approach for grading hip OA severity from CT images was developed. The models have shown comparable performance with high ONCA, which facilitates automated grading in large-scale CT databases and indicates the potential for further disease progression analysis. Classification accuracy was correlated with the model uncertainty, which would allow for the prediction of classification errors.
△ Less
Submitted 30 December, 2023;
originally announced January 2024.
-
AI agents for facilitating social interactions and wellbeing
Authors:
Hiro Taiyo Hamada,
Ryota Kanai
Abstract:
Wellbeing AI has been becoming a new trend in individuals' mental health, organizational health, and flourishing our societies. Various applications of wellbeing AI have been introduced to our daily lives. While social relationships within groups are a critical factor for wellbeing, the development of wellbeing AI for social interactions remains relatively scarce. In this paper, we provide an over…
▽ More
Wellbeing AI has been becoming a new trend in individuals' mental health, organizational health, and flourishing our societies. Various applications of wellbeing AI have been introduced to our daily lives. While social relationships within groups are a critical factor for wellbeing, the development of wellbeing AI for social interactions remains relatively scarce. In this paper, we provide an overview of the mediative role of AI-augmented agents for social interactions. First, we discuss the two-dimensional framework for classifying wellbeing AI: individual/group and analysis/intervention. Furthermore, wellbeing AI touches on intervening social relationships between human-human interactions since positive social relationships are key to human wellbeing. This intervention may raise technical and ethical challenges. We discuss opportunities and challenges of the relational approach with wellbeing AI to promote wellbeing in our societies.
△ Less
Submitted 25 February, 2022;
originally announced March 2022.
-
Mathematics in Caging of Robotics
Authors:
Hiroyasu Hamada,
Satoshi Makita,
Shigeki Matsutani
Abstract:
It is a crucial problem in robotics field to cage an object using robots like multifingered hand. However the problem what is the caging for general geometrical objects and robots has not been well-described in mathematics though there were many rigorous studies on the methods how to cage an object by certain robots. In this article, we investigate the caging problem more mathematically and descri…
▽ More
It is a crucial problem in robotics field to cage an object using robots like multifingered hand. However the problem what is the caging for general geometrical objects and robots has not been well-described in mathematics though there were many rigorous studies on the methods how to cage an object by certain robots. In this article, we investigate the caging problem more mathematically and describe the problem in terms of recursion of the simple euclidean moves. Using the description, we show that the caging has the degree of difficulty which is closely related to a combinatorial problem and a wire puzzle. It implies that in order to capture an object by caging, from a practical viewpoint the difficulty plays an important role.
△ Less
Submitted 6 January, 2017;
originally announced January 2017.
-
Image transmission over OFDM channel with rate allocation scheme and minimum peak-toaverage power ratio
Authors:
Usama S. Mohammed,
H. A. Hamada
Abstract:
This paper proposes new scheme for efficient rate allocation in conjunction with reducing peak-to-average power ratio (PAPR) in orthogonal frequency-division multiplexing (OFDM) systems. Modification of the set partitioning in hierarchical trees (SPIHT) image coder is proposed to generate four different groups of bit-stream relative to its significances. The significant bits, the sign bits, the se…
▽ More
This paper proposes new scheme for efficient rate allocation in conjunction with reducing peak-to-average power ratio (PAPR) in orthogonal frequency-division multiplexing (OFDM) systems. Modification of the set partitioning in hierarchical trees (SPIHT) image coder is proposed to generate four different groups of bit-stream relative to its significances. The significant bits, the sign bits, the set bits and the refinement bits are transmitted in four different groups. The proposed method for reducing the PAPR utilizes twice the unequal error protection (UEP), using the Read-Solomon codes (RS), in conjunction with bit-rate allocation and selective interleaving to provide minimum PAPR. The output bit-stream from the source code (SPIHT) will be started by the most significant types of bits (first group of bits). The optimal unequal error protection (UEP) of the four groups is proposed based on the channel destortion. The proposed structure provides significant improvement in bit error rate (BER) performance. Performed computer simulations have shown that the proposed scheme outperform the performance of most of the recent PAPR reduction techniques in most cases. Moreover, the simulation results indicate that the proposed scheme provides significantly better PSNR performance in comparison to well-known robust coding schemes.
△ Less
Submitted 4 June, 2010;
originally announced June 2010.
