-
Reconstructing Depth Images of Moving Objects from Wi-Fi CSI Data
Authors:
Guanyu Cao,
Takuya Maekawa,
Kazuya Ohara,
Yasue Kishino
Abstract:
This study proposes a new deep learning method for reconstructing depth images of moving objects within a specific area using Wi-Fi channel state information (CSI). The Wi-Fi-based depth imaging technique has novel applications in domains such as security and elder care. However, reconstructing depth images from CSI is challenging because learning the mapping function between CSI and depth images,…
▽ More
This study proposes a new deep learning method for reconstructing depth images of moving objects within a specific area using Wi-Fi channel state information (CSI). The Wi-Fi-based depth imaging technique has novel applications in domains such as security and elder care. However, reconstructing depth images from CSI is challenging because learning the mapping function between CSI and depth images, both of which are high-dimensional data, is particularly difficult. To address the challenge, we propose a new approach called Wi-Depth. The main idea behind the design of Wi-Depth is that a depth image of a moving object can be decomposed into three core components: the shape, depth, and position of the target. Therefore, in the depth-image reconstruction task, Wi-Depth simultaneously estimates the three core pieces of information as auxiliary tasks in our proposed VAE-based teacher-student architecture, enabling it to output images with the consistency of a correct shape, depth, and position. In addition, the design of Wi-Depth is based on our idea that this decomposition efficiently takes advantage of the fact that shape, depth, and position relate to primitive information inferred from CSI such as angle-of-arrival, time-of-flight, and Doppler frequency shift.
△ Less
Submitted 9 March, 2025;
originally announced March 2025.
-
LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs
Authors:
LLM-jp,
:,
Akiko Aizawa,
Eiji Aramaki,
Bowen Chen,
Fei Cheng,
Hiroyuki Deguchi,
Rintaro Enomoto,
Kazuki Fujii,
Kensuke Fukumoto,
Takuya Fukushima,
Namgi Han,
Yuto Harada,
Chikara Hashimoto,
Tatsuya Hiraoka,
Shohei Hisada,
Sosuke Hosokawa,
Lu Jie,
Keisuke Kamata,
Teruhito Kanazawa,
Hiroki Kanezashi,
Hiroshi Kataoka,
Satoru Katsumata,
Daisuke Kawahara,
Seiya Kawano
, et al. (58 additional authors not shown)
Abstract:
This paper introduces LLM-jp, a cross-organizational project for the research and development of Japanese large language models (LLMs). LLM-jp aims to develop open-source and strong Japanese LLMs, and as of this writing, more than 1,500 participants from academia and industry are working together for this purpose. This paper presents the background of the establishment of LLM-jp, summaries of its…
▽ More
This paper introduces LLM-jp, a cross-organizational project for the research and development of Japanese large language models (LLMs). LLM-jp aims to develop open-source and strong Japanese LLMs, and as of this writing, more than 1,500 participants from academia and industry are working together for this purpose. This paper presents the background of the establishment of LLM-jp, summaries of its activities, and technical reports on the LLMs developed by LLM-jp. For the latest activities, visit https://llm-jp.nii.ac.jp/en/.
△ Less
Submitted 30 December, 2024; v1 submitted 4 July, 2024;
originally announced July 2024.
-
Preliminary Investigation of SSL for Complex Work Activity Recognition in Industrial Domain via MoIL
Authors:
Qingxin Xia,
Takuya Maekawa,
Jaime Morales,
Takahiro Hara,
Hirotomo Oshima,
Masamitsu Fukuda,
Yasuo Namioka
Abstract:
In this study, we investigate a new self-supervised learning (SSL) approach for complex work activity recognition using wearable sensors. Owing to the cost of labeled sensor data collection, SSL methods for human activity recognition (HAR) that effectively use unlabeled data for pretraining have attracted attention. However, applying prior SSL to complex work activities such as packaging works is…
▽ More
In this study, we investigate a new self-supervised learning (SSL) approach for complex work activity recognition using wearable sensors. Owing to the cost of labeled sensor data collection, SSL methods for human activity recognition (HAR) that effectively use unlabeled data for pretraining have attracted attention. However, applying prior SSL to complex work activities such as packaging works is challenging because the observed data vary considerably depending on situations such as the number of items to pack and the size of the items in the case of packaging works. In this study, we focus on sensor data corresponding to characteristic and necessary actions (sensor data motifs) in a specific activity such as a stretching packing tape action in an assembling a box activity, and \textcolor{black}{try} to train a neural network in self-supervised learning so that it identifies occurrences of the characteristic actions, i.e., Motif Identification Learning (MoIL). The feature extractor in the network is used in the downstream task, i.e., work activity recognition, enabling precise activity recognition containing characteristic actions with limited labeled training data. The MoIL approach was evaluated on real-world work activity data and it achieved state-of-the-art performance under limited training labels.
△ Less
Submitted 21 April, 2024;
originally announced April 2024.
-
A Summarized History-based Dialogue System for Amnesia-Free Prompt Updates
Authors:
Hyejin Hong,
Hibiki Kawano,
Takuto Maekawa,
Naoki Yoshimaru,
Takamasa Iio,
Kenji Hatano
Abstract:
In today's society, information overload presents challenges in providing optimal recommendations. Consequently, the importance of dialogue systems that can discern and provide the necessary information through dialogue is increasingly recognized. However, some concerns existing dialogue systems rely on pre-trained models and need help to cope with real-time or insufficient information. To address…
▽ More
In today's society, information overload presents challenges in providing optimal recommendations. Consequently, the importance of dialogue systems that can discern and provide the necessary information through dialogue is increasingly recognized. However, some concerns existing dialogue systems rely on pre-trained models and need help to cope with real-time or insufficient information. To address these concerns, models that allow the addition of missing information to dialogue robots are being proposed. Yet, maintaining the integrity of previous conversation history while integrating new data remains a formidable challenge. This paper presents a novel system for dialogue robots designed to remember user-specific characteristics by retaining past conversation history even as new information is added.
△ Less
Submitted 21 December, 2023;
originally announced December 2023.
-
Collect and Connect Data Leaves to Feature Concepts: Interactive Graph Generation Toward Well-being
Authors:
Yukio Ohsawa,
Tomohide Maekawa,
Hiroki Yamaguchi,
Hiro Yoshida,
Kaira Sekiguchi
Abstract:
Feature concepts and data leaves have been invented using datasets to foster creative thoughts for creating well-being in daily life. The idea, simply put, is to attach selected and collected data leaves that are summaries of event flows to be discovered from corresponding datasets, on the target feature concept representing the well-being aimed. A graph of existing or expected datasets to be atta…
▽ More
Feature concepts and data leaves have been invented using datasets to foster creative thoughts for creating well-being in daily life. The idea, simply put, is to attach selected and collected data leaves that are summaries of event flows to be discovered from corresponding datasets, on the target feature concept representing the well-being aimed. A graph of existing or expected datasets to be attached to a feature concept is generated semi-automatically. Rather than sheer automated generative AI, our work addresses the process of generative artificial and natural intelligence to create the basis for data use and reuse.
△ Less
Submitted 16 December, 2023;
originally announced December 2023.
-
Unsupervised Human Activity Recognition through Two-stage Prompting with ChatGPT
Authors:
Qingxin Xia,
Takuya Maekawa,
Takahiro Hara
Abstract:
Wearable sensor devices, which offer the advantage of recording daily objects used by a person while performing an activity, enable the feasibility of unsupervised Human Activity Recognition (HAR). Unfortunately, previous unsupervised approaches using the usage sequence of objects usually require a proper description of activities manually prepared by humans. Instead, we leverage the knowledge emb…
▽ More
Wearable sensor devices, which offer the advantage of recording daily objects used by a person while performing an activity, enable the feasibility of unsupervised Human Activity Recognition (HAR). Unfortunately, previous unsupervised approaches using the usage sequence of objects usually require a proper description of activities manually prepared by humans. Instead, we leverage the knowledge embedded in a Large Language Model (LLM) of ChatGPT. Because the sequence of objects robustly characterizes the activity identity, it is possible that ChatGPT already learned the association between activities and objects from existing contexts. However, previous prompt engineering for ChatGPT exhibits limited generalization ability when dealing with a list of words (i.e., sequence of objects) due to the similar weighting assigned to each word in the list. In this study, we propose a two-stage prompt engineering, which first guides ChatGPT to generate activity descriptions associated with objects while emphasizing important objects for distinguishing similar activities; then outputs activity classes and explanations for enhancing the contexts that are helpful for HAR. To the best of our knowledge, this is the first study that utilizes ChatGPT to recognize activities using objects in an unsupervised manner. We conducted our approach on three datasets and demonstrated the state-of-the-art performance.
△ Less
Submitted 3 June, 2023;
originally announced June 2023.
-
A benchmark for computational analysis of animal behavior, using animal-borne tags
Authors:
Benjamin Hoffman,
Maddie Cusimano,
Vittorio Baglione,
Daniela Canestrari,
Damien Chevallier,
Dominic L. DeSantis,
Lorène Jeantet,
Monique A. Ladds,
Takuya Maekawa,
Vicente Mata-Silva,
Víctor Moreno-González,
Anthony Pagano,
Eva Trapote,
Outi Vainio,
Antti Vehkaoja,
Ken Yoda,
Katherine Zacarian,
Ari Friedlaender
Abstract:
Animal-borne sensors (`bio-loggers') can record a suite of kinematic and environmental data, which are used to elucidate animal ecophysiology and improve conservation efforts. Machine learning techniques are used for interpreting the large amounts of data recorded by bio-loggers, but there exists no common framework for comparing the different machine learning techniques in this domain. This makes…
▽ More
Animal-borne sensors (`bio-loggers') can record a suite of kinematic and environmental data, which are used to elucidate animal ecophysiology and improve conservation efforts. Machine learning techniques are used for interpreting the large amounts of data recorded by bio-loggers, but there exists no common framework for comparing the different machine learning techniques in this domain. This makes it difficult to, for example, identify patterns in what works well for machine learning-based analysis of bio-logger data. It also makes it difficult to evaluate the effectiveness of novel methods developed by the machine learning community.
To address this, we present the Bio-logger Ethogram Benchmark (BEBE), a collection of datasets with behavioral annotations, as well as a modeling task and evaluation metrics. BEBE is to date the largest, most taxonomically diverse, publicly available benchmark of this type. Using BEBE, we compare the performance of deep and classical machine learning methods for identifying animal behaviors based on bio-logger data. As an example usage of BEBE, we test an approach based on self-supervised learning. To apply this approach to animal behavior classification, we adapt a deep neural network pre-trained with 700,000 hours of data collected from human wrist-worn accelerometers.
We find that deep neural networks out-perform the classical machine learning methods we tested across all nine datasets in BEBE. We additionally find that the approach based on self-supervised learning out-performs the alternatives we tested, especially in settings when there is a low amount of training data available. In light of this, we are able to make concrete suggestions for designing studies that rely on machine learning to infer behavior from bio-logger data. Datasets and code are available at https://github.com/earthspecies/BEBE.
△ Less
Submitted 27 September, 2024; v1 submitted 18 May, 2023;
originally announced May 2023.
-
OpenPack: A Large-scale Dataset for Recognizing Packaging Works in IoT-enabled Logistic Environments
Authors:
Naoya Yoshimura,
Jaime Morales,
Takuya Maekawa,
Takahiro Hara
Abstract:
Unlike human daily activities, existing publicly available sensor datasets for work activity recognition in industrial domains are limited by difficulties in collecting realistic data as close collaboration with industrial sites is required. This also limits research on and development of methods for industrial applications. To address these challenges and contribute to research on machine recogni…
▽ More
Unlike human daily activities, existing publicly available sensor datasets for work activity recognition in industrial domains are limited by difficulties in collecting realistic data as close collaboration with industrial sites is required. This also limits research on and development of methods for industrial applications. To address these challenges and contribute to research on machine recognition of work activities in industrial domains, in this study, we introduce a new large-scale dataset for packaging work recognition called OpenPack. OpenPack contains 53.8 hours of multimodal sensor data, including acceleration data, keypoints, depth images, and readings from IoT-enabled devices (e.g., handheld barcode scanners), collected from 16 distinct subjects with different levels of packaging work experience. We apply state-of-the-art human activity recognition techniques to the dataset and provide future directions of complex work activity recognition studies in the pervasive computing community based on the results. We believe that OpenPack will contribute to the sensor-based action/activity recognition community by providing challenging tasks. The OpenPack dataset is available at https://open-pack.github.io.
△ Less
Submitted 20 April, 2024; v1 submitted 10 December, 2022;
originally announced December 2022.
-
Data Leaves: Scenario-oriented Metadata for Data Federative Innovation
Authors:
Yukio Ohsawa,
Kaira Sekiguchi,
Tomohide Maekawa,
Hiroki Yamaguchi,
Son Yeon Hyuk,
Sae Kondo
Abstract:
A method for representing the digest information of each dataset is proposed, oriented to the aid of innovative thoughts and the communication of data users who attempt to create valuable products, services, and business models using or combining datasets. Compared with methods for connecting datasets via shared attributes (i.e., variables), this method connects datasets via events, situations, or…
▽ More
A method for representing the digest information of each dataset is proposed, oriented to the aid of innovative thoughts and the communication of data users who attempt to create valuable products, services, and business models using or combining datasets. Compared with methods for connecting datasets via shared attributes (i.e., variables), this method connects datasets via events, situations, or actions in a scenario that is supposed to be active in the real world. This method reflects the consideration of the fitness of each metadata to the feature concept, which is an abstract of the information or knowledge expected to be acquired from data; thus, the users of the data acquire practical knowledge that fits the requirements of real businesses and real life, as well as grounds for realistic application of AI technologies to data.
△ Less
Submitted 7 August, 2022;
originally announced August 2022.
-
Using Social Media Background to Improve Cold-start Recommendation Deep Models
Authors:
Yihong Zhang,
Takuya Maekawa,
Takahiro Hara
Abstract:
In recommender systems, a cold-start problem occurs when there is no past interaction record associated with the user or item. Typical solutions to the cold-start problem make use of contextual information, such as user demographic attributes or product descriptions. A group of works have shown that social media background can help predicting temporal phenomenons such as product sales and stock pr…
▽ More
In recommender systems, a cold-start problem occurs when there is no past interaction record associated with the user or item. Typical solutions to the cold-start problem make use of contextual information, such as user demographic attributes or product descriptions. A group of works have shown that social media background can help predicting temporal phenomenons such as product sales and stock price movements. In this work, our goal is to investigate whether social media background can be used as extra contextual information to improve recommendation models. Based on an existing deep neural network model, we proposed a method to represent temporal social media background as embeddings and fuse them as an extra component in the model. We conduct experimental evaluations on a real-world e-commerce dataset and a Twitter dataset. The results show that our method of fusing social media background with the existing model does generally improve recommendation performance. In some cases the recommendation accuracy measured by hit-rate@K doubles after fusing with social media background. Our findings can be beneficial for future recommender system designs that consider complex temporal information representing social interests.
△ Less
Submitted 4 June, 2021;
originally announced June 2021.
-
Privacy-Preserving Support Vector Machine Computing Using Random Unitary Transformation
Authors:
Takahiro Maekawa,
Ayana Kawamura,
Takayuki Nakachi,
Hitoshi Kiya
Abstract:
A privacy-preserving support vector machine (SVM) computing scheme is proposed in this paper. Cloud computing has been spreading in many fields. However, the cloud computing has some serious issues for end users, such as the unauthorized use of cloud services, data leaks, and privacy being compromised. Accordingly, we consider privacy-preserving SVM computing. We focus on protecting visual \red{in…
▽ More
A privacy-preserving support vector machine (SVM) computing scheme is proposed in this paper. Cloud computing has been spreading in many fields. However, the cloud computing has some serious issues for end users, such as the unauthorized use of cloud services, data leaks, and privacy being compromised. Accordingly, we consider privacy-preserving SVM computing. We focus on protecting visual \red{information} of images by using a random unitary transformation. Some properties of the protected images are discussed. The proposed scheme enables us not only to protect images, but also to have the same performance as that of unprotected images even when using typical kernel functions such as the linear kernel, radial basis function(RBF) kernel and polynomial kernel. Moreover, it can be directly carried out by using well-known SVM algorithms, without preparing any algorithms specialized for secure SVM computing. In an experiment, the proposed scheme is applied to a face-based authentication algorithm with SVM classifiers to confirm the effectiveness.
△ Less
Submitted 20 August, 2019;
originally announced August 2019.
-
Privacy-Preserving Deep Neural Networks with Pixel-based Image Encryption Considering Data Augmentation in the Encrypted Domain
Authors:
Warit Sirichotedumrong,
Takahiro Maekawa,
Yuma Kinoshita,
Hitoshi Kiya
Abstract:
We present a novel privacy-preserving scheme for deep neural networks (DNNs) that enables us not to only apply images without visual information to DNNs for both training and testing but to also consider data augmentation in the encrypted domain for the first time. In this paper, a novel pixel-based image encryption method is first proposed for privacy-preserving DNNs. In addition, a novel adaptat…
▽ More
We present a novel privacy-preserving scheme for deep neural networks (DNNs) that enables us not to only apply images without visual information to DNNs for both training and testing but to also consider data augmentation in the encrypted domain for the first time. In this paper, a novel pixel-based image encryption method is first proposed for privacy-preserving DNNs. In addition, a novel adaptation network is considered that reduces the influence of image encryption. In an experiment, the proposed method is applied to a well-known network, ResNet-18, for image classification. The experimental results demonstrate that conventional privacy-preserving machine learning methods including the state-of-the-arts cannot be applied to data augmentation in the encrypted domain and that the proposed method outperforms them in terms of classification accuracy.
△ Less
Submitted 6 May, 2019;
originally announced May 2019.
-
Privacy-Preserving SVM Computing by Using Random Unitary Transformation
Authors:
Takahiro Maekawa,
Takayuki Nakachi,
Sayaka Shiota,
Hitoshi Kiya
Abstract:
A privacy-preserving Support Vector Machine (SVM) computing scheme is proposed in this paper. Cloud computing has been spreading in many fields. However, the cloud computing has some serious issues for end users, such as unauthorized use and leak of data, and privacy compromise. We focus on templates protected by using a random unitary transformation, and consider some properties of the protected…
▽ More
A privacy-preserving Support Vector Machine (SVM) computing scheme is proposed in this paper. Cloud computing has been spreading in many fields. However, the cloud computing has some serious issues for end users, such as unauthorized use and leak of data, and privacy compromise. We focus on templates protected by using a random unitary transformation, and consider some properties of the protected templates for secure SVM computing, where templates mean features extracted from data. The proposed scheme enables us not only to protect templates, but also to have the same performance as that of unprotected templates under some useful kernel functions. Moreover, it can be directly carried out by using well-known SVM algorithms, without preparing any algorithms specialized for secure SVM computing. In the experiments, the proposed scheme is applied to a face-based authentication algorithm with SVM classifiers to confirm the effectiveness.
△ Less
Submitted 19 September, 2018;
originally announced September 2018.
