-
PLaMo-100B: A Ground-Up Language Model Designed for Japanese Proficiency
Authors:
Preferred Elements,
:,
Kenshin Abe,
Kaizaburo Chubachi,
Yasuhiro Fujita,
Yuta Hirokawa,
Kentaro Imajo,
Toshiki Kataoka,
Hiroyoshi Komatsu,
Hiroaki Mikami,
Tsuguo Mogami,
Shogo Murai,
Kosuke Nakago,
Daisuke Nishino,
Toru Ogawa,
Daisuke Okanohara,
Yoshihiko Ozaki,
Shotaro Sano,
Shuji Suzuki,
Tianqi Xu,
Toshihiko Yanase
Abstract:
We introduce PLaMo-100B, a large-scale language model designed for Japanese proficiency. The model was trained from scratch using 2 trillion tokens, with architecture such as QK Normalization and Z-Loss to ensure training stability during the training process. Post-training techniques, including Supervised Fine-Tuning and Direct Preference Optimization, were applied to refine the model's performan…
▽ More
We introduce PLaMo-100B, a large-scale language model designed for Japanese proficiency. The model was trained from scratch using 2 trillion tokens, with architecture such as QK Normalization and Z-Loss to ensure training stability during the training process. Post-training techniques, including Supervised Fine-Tuning and Direct Preference Optimization, were applied to refine the model's performance. Benchmark evaluations suggest that PLaMo-100B performs well, particularly in Japanese-specific tasks, achieving results that are competitive with frontier models like GPT-4. The base model is available at https://huggingface.co/pfnet/plamo-100b.
△ Less
Submitted 22 October, 2024; v1 submitted 9 October, 2024;
originally announced October 2024.
-
Behavioral assessment of a humanoid robot when attracting pedestrians in a mall
Authors:
Yuki Okafuji,
Yasunori Ozaki,
Jun Baba,
Junya Nakanishi,
Kohei Ogawa,
Yuichiro Yoshikawa,
Hiroshi Ishiguro
Abstract:
Research currently being conducted on the use of robots as human labor support technology. In particular, the service industry needs to allocate more manpower, and it will be important for robots to support people. This study focuses on using a humanoid robot as a social service robot to convey information in a shopping mall, and the robot's behavioral concepts were analyzed. In order to convey th…
▽ More
Research currently being conducted on the use of robots as human labor support technology. In particular, the service industry needs to allocate more manpower, and it will be important for robots to support people. This study focuses on using a humanoid robot as a social service robot to convey information in a shopping mall, and the robot's behavioral concepts were analyzed. In order to convey the information, two processes must occur. Pedestrians must stop in front of the robot, and the robot must continue the engagement with them. For the purpose of this study, three types of autonomous behavioral concepts of the robot for the general use were analyzed and compared in these processes in the experiment: active, passive-negative, and passive-positive concepts. After interactions were attempted with 65,000+ pedestrians, this study revealed that the passive-negative concept can make pedestrians stop more and stay longer. In order to evaluate the effectiveness of the robot in a real environment, the comparative results between three behaviors and human advertisers revealed that (1) the results of the active and passive-positive concepts of the robot are comparable to those of the humans, and (2) the performance of the passive-negative concept is higher than that of all participants. These findings demonstrate that the performance of robots is comparable to that of humans in providing information tasks in a limited environment; therefore, it is expected that service robots as a labor support technology will be able to perform well in the real world.
△ Less
Submitted 6 September, 2021;
originally announced September 2021.
-
3D Head-Position Prediction in First-Person View by Considering Head Pose for Human-Robot Eye Contact
Authors:
Yuki Tamaru,
Yasunori Ozaki,
Yuki Okafuji,
Junya Nakanishi,
Yuichiro Yoshikawa,
Jun Baba
Abstract:
For a humanoid robot to make eye contact and initiate communication with a person, it is necessary to estimate the person's head position. However, eye contact becomes difficult due to the mechanical delay of the robot when the person is moving. Owing to these issues, it is important to conduct a head-position prediction to mitigate the effect of the delay in the robot motion. Based on the fact th…
▽ More
For a humanoid robot to make eye contact and initiate communication with a person, it is necessary to estimate the person's head position. However, eye contact becomes difficult due to the mechanical delay of the robot when the person is moving. Owing to these issues, it is important to conduct a head-position prediction to mitigate the effect of the delay in the robot motion. Based on the fact that humans turn their heads before changing direction while walking, we hypothesized that the accuracy of three-dimensional (3D) head-position prediction from a first-person view can be improved by considering the head pose. We compared our method with a conventional Kalman filter-based approach, and found our method to be more accurate. The experiment results show that considering the head pose helps improve the accuracy of 3D head-position prediction.
△ Less
Submitted 20 January, 2022; v1 submitted 10 March, 2021;
originally announced March 2021.
-
Warm Starting CMA-ES for Hyperparameter Optimization
Authors:
Masahiro Nomura,
Shuhei Watanabe,
Youhei Akimoto,
Yoshihiko Ozaki,
Masaki Onishi
Abstract:
Hyperparameter optimization (HPO), formulated as black-box optimization (BBO), is recognized as essential for automation and high performance of machine learning approaches. The CMA-ES is a promising BBO approach with a high degree of parallelism, and has been applied to HPO tasks, often under parallel implementation, and shown superior performance to other approaches including Bayesian optimizati…
▽ More
Hyperparameter optimization (HPO), formulated as black-box optimization (BBO), is recognized as essential for automation and high performance of machine learning approaches. The CMA-ES is a promising BBO approach with a high degree of parallelism, and has been applied to HPO tasks, often under parallel implementation, and shown superior performance to other approaches including Bayesian optimization (BO). However, if the budget of hyperparameter evaluations is severely limited, which is often the case for end users who do not deserve parallel computing, the CMA-ES exhausts the budget without improving the performance due to its long adaptation phase, resulting in being outperformed by BO approaches. To address this issue, we propose to transfer prior knowledge on similar HPO tasks through the initialization of the CMA-ES, leading to significantly shortening the adaptation time. The knowledge transfer is designed based on the novel definition of task similarity, with which the correlation of the performance of the proposed approach is confirmed on synthetic problems. The proposed warm starting CMA-ES, called WS-CMA-ES, is applied to different HPO tasks where some prior knowledge is available, showing its superior performance over the original CMA-ES as well as BO approaches with or without using the prior knowledge.
△ Less
Submitted 12 December, 2020;
originally announced December 2020.
-
Can User-Centered Reinforcement Learning Allow a Robot to Attract Passersby without Causing Discomfort?
Authors:
Yasunori Ozaki,
Tatsuya Ishihara,
Narimune Matsumura,
Tadashi Nunobiki
Abstract:
The aim of our study was to develop a method by which a social robot can greet passersby and get their attention without causing them to suffer discomfort.A number of customer services have recently come to be provided by social robots rather than people, including, serving as receptionists, guides, and exhibitors. Robot exhibitors, for example, can explain products being promoted by the robot own…
▽ More
The aim of our study was to develop a method by which a social robot can greet passersby and get their attention without causing them to suffer discomfort.A number of customer services have recently come to be provided by social robots rather than people, including, serving as receptionists, guides, and exhibitors. Robot exhibitors, for example, can explain products being promoted by the robot owners. However, a sudden greeting by a robot can startle passersby and cause discomfort to passersby.Social robots should thus adapt their mannerisms to the situation they face regarding passersby.We developed a method for meeting this requirement on the basis of the results of related work. Our proposed method, user-centered reinforcement learning, enables robots to greet passersby and get their attention without causing them to suffer discomfort (p<0.01) .The results of an experiment in the field, an office entrance, demonstrated that our method meets this requirement.
△ Less
Submitted 1 January, 2020; v1 submitted 14 March, 2019;
originally announced March 2019.
