Showing 1–50 of 85 results for author: Sato, T

Full-body WPT: wireless powering with meandered e-textiles

Authors: Ryo Takahashi, Takashi Sato, Wakako Yukita, Tomoyuki Yokota, Takao Someya, Yoshihiro Kawahara

Abstract: We present Full-body WPT, wireless power networking around the human body using a meandered textile coil. Unlike traditional inductive systems that emit strong fields into the deep tissue inside the body, the meander coil enables localized generation of strong magnetic field constrained to the skin surface, even when scaled to the size of the human body. Such localized inductive system enhances bo… ▽ More We present Full-body WPT, wireless power networking around the human body using a meandered textile coil. Unlike traditional inductive systems that emit strong fields into the deep tissue inside the body, the meander coil enables localized generation of strong magnetic field constrained to the skin surface, even when scaled to the size of the human body. Such localized inductive system enhances both safety and efficiency of wireless power around the body. Furthermore, the use of low-loss conductive yarn achieve energy-efficient and lightweight design. We analyze the performance of our design through simulations and experimental prototypes, demonstrating high power transfer efficiency and adaptability to user movement and posture. Our system provides a safe and efficient distributed power network using meandered textile coils integrated into wearable materials, highlighting the potential of body-centric wireless power networking as a foundational layer for ubiquitous health monitoring, augmented reality, and human-machine interaction systems. △ Less

Submitted 21 June, 2025; originally announced June 2025.

Lookup Table-based Multiplication-free All-digital DNN Accelerator Featuring Self-Synchronous Pipeline Accumulation

Authors: Hiroto Tagata, Takashi Sato, Hiromitsu Awano

Abstract: Deep neural networks (DNNs) have been widely applied in our society, yet reducing power consumption due to large-scale matrix computations remains a critical challenge. MADDNESS is a known approach to improving energy efficiency by substituting matrix multiplication with table lookup operations. Previous research has employed large analog computing circuits to convert inputs into LUT addresses, wh… ▽ More Deep neural networks (DNNs) have been widely applied in our society, yet reducing power consumption due to large-scale matrix computations remains a critical challenge. MADDNESS is a known approach to improving energy efficiency by substituting matrix multiplication with table lookup operations. Previous research has employed large analog computing circuits to convert inputs into LUT addresses, which presents challenges to area efficiency and computational accuracy. This paper proposes a novel MADDNESS-based all-digital accelerator featuring a self-synchronous pipeline accumulator, resulting in a compact, energy-efficient, and PVT-invariant computation. Post-layout simulation using a commercial 22nm process showed that 2.5 times higher energy efficiency (174 TOPS/W) and 5 times higher area efficiency (2.01 TOPS/mm2) can be achieved compared to the conventional accelerator. △ Less

Submitted 20 June, 2025; originally announced June 2025.

Interior-Point Vanishing Problem in Semidefinite Relaxations for Neural Network Verification

Authors: Ryota Ueda, Takami Sato, Ken Kobayashi, Kazuhide Nakata

Abstract: Semidefinite programming (SDP) relaxation has emerged as a promising approach for neural network verification, offering tighter bounds than other convex relaxation methods for deep neural networks (DNNs) with ReLU activations. However, we identify a critical limitation in the SDP relaxation when applied to deep networks: interior-point vanishing, which leads to the loss of strict feasibility -- a… ▽ More Semidefinite programming (SDP) relaxation has emerged as a promising approach for neural network verification, offering tighter bounds than other convex relaxation methods for deep neural networks (DNNs) with ReLU activations. However, we identify a critical limitation in the SDP relaxation when applied to deep networks: interior-point vanishing, which leads to the loss of strict feasibility -- a crucial condition for the numerical stability and optimality of SDP. Through rigorous theoretical and empirical analysis, we demonstrate that as the depth of DNNs increases, the strict feasibility is likely to be lost, creating a fundamental barrier to scaling SDP-based verification. To address the interior-point vanishing, we design and investigate five solutions to enhance the feasibility conditions of the verification problem. Our methods can successfully solve 88% of the problems that could not be solved by existing methods, accounting for 41% of the total. Our analysis also reveals that the valid constraints for the lower and upper bounds for each ReLU unit are traditionally inherited from prior work without solid reasons, but are actually not only unbeneficial but also even harmful to the problem's feasibility. This work provides valuable insights into the fundamental challenges of SDP-based DNN verification and offers practical solutions to improve its applicability to deeper neural networks, contributing to the development of more reliable and secure systems with DNNs. △ Less

Submitted 11 June, 2025; originally announced June 2025.

Comments: 17 pages, 2 figures. Version revised after ICML 2025 reviews

MSC Class: 90C22; 68T05 ACM Class: I.2.6; G.1.6

Fast Parameter Optimization of Delayed Feedback Reservoir with Backpropagation and Gradient Descent

Authors: Sosei Ikeda, Hiromitsu Awano, Takashi Sato

Abstract: A delayed feedback reservoir (DFR) is a reservoir computing system well-suited for hardware implementations. However, achieving high accuracy in DFRs depends heavily on selecting appropriate hyperparameters. Conventionally, due to the presence of a non-linear circuit block in the DFR, the grid search has only been the preferred method, which is computationally intensive and time-consuming and thus… ▽ More A delayed feedback reservoir (DFR) is a reservoir computing system well-suited for hardware implementations. However, achieving high accuracy in DFRs depends heavily on selecting appropriate hyperparameters. Conventionally, due to the presence of a non-linear circuit block in the DFR, the grid search has only been the preferred method, which is computationally intensive and time-consuming and thus performed offline. This paper presents a fast and accurate parameter optimization method for DFRs. To this end, we leverage the well-known backpropagation and gradient descent framework with the state-of-the-art DFR model for the first time to facilitate parameter optimization. We further propose a truncated backpropagation strategy applicable to the recursive dot-product reservoir representation to achieve the highest accuracy with reduced memory usage. With the proposed lightweight implementation, the computation time has been significantly reduced by up to 1/700 of the grid search. △ Less

Submitted 16 April, 2025; originally announced April 2025.

Comments: arXiv admin note: substantial text overlap with arXiv:2504.11970

Hardware-Friendly Delayed-Feedback Reservoir for Multivariate Time-Series Classification

Authors: Sosei Ikeda, Hiromitsu Awano, Takashi Sato

Abstract: Reservoir computing (RC) is attracting attention as a machine-learning technique for edge computing. In time-series classification tasks, the number of features obtained using a reservoir depends on the length of the input series. Therefore, the features must be converted to a constant-length intermediate representation (IR), such that they can be processed by an output layer. Existing conversion… ▽ More Reservoir computing (RC) is attracting attention as a machine-learning technique for edge computing. In time-series classification tasks, the number of features obtained using a reservoir depends on the length of the input series. Therefore, the features must be converted to a constant-length intermediate representation (IR), such that they can be processed by an output layer. Existing conversion methods involve computationally expensive matrix inversion that significantly increases the circuit size and requires processing power when implemented in hardware. In this article, we propose a simple but effective IR, namely, dot-product-based reservoir representation (DPRR), for RC based on the dot product of data features. Additionally, we propose a hardware-friendly delayed-feedback reservoir (DFR) consisting of a nonlinear element and delayed feedback loop with DPRR. The proposed DFR successfully classified multivariate time series data that has been considered particularly difficult to implement efficiently in hardware. In contrast to conventional DFR models that require analog circuits, the proposed model can be implemented in a fully digital manner suitable for high-level syntheses. A comparison with existing machine-learning methods via field-programmable gate array implementation using 12 multivariate time-series classification tasks confirmed the superior accuracy and small circuit size of the proposed method. △ Less

Submitted 16 April, 2025; originally announced April 2025.

Online Training and Inference System on Edge FPGA Using Delayed Feedback Reservoir

Authors: Sosei Ikeda, Hiromitsu Awano, Takashi Sato

Abstract: A delayed feedback reservoir (DFR) is a hardwarefriendly reservoir computing system. Implementing DFRs in embedded hardware requires efficient online training. However, two main challenges prevent this: hyperparameter selection, which is typically done by offline grid search, and training of the output linear layer, which is memory-intensive. This paper introduces a fast and accurate parameter opt… ▽ More A delayed feedback reservoir (DFR) is a hardwarefriendly reservoir computing system. Implementing DFRs in embedded hardware requires efficient online training. However, two main challenges prevent this: hyperparameter selection, which is typically done by offline grid search, and training of the output linear layer, which is memory-intensive. This paper introduces a fast and accurate parameter optimization method for the reservoir layer utilizing backpropagation and gradient descent by adopting a modular DFR model. A truncated backpropagation strategy is proposed to reduce memory consumption associated with the expansion of the recursive structure while maintaining accuracy. The computation time is significantly reduced compared to grid search. Additionally, an in-place Ridge regression for the output layer via 1-D Cholesky decomposition is presented, reducing memory usage to be 1/4. These methods enable the realization of an online edge training and inference system of DFR on an FPGA, reducing computation time by about 1/13 and power consumption by about 1/27 compared to software implementation on the same board. △ Less

Submitted 16 April, 2025; originally announced April 2025.

SLAMSpoof: Practical LiDAR Spoofing Attacks on Localization Systems Guided by Scan Matching Vulnerability Analysis

Authors: Rokuto Nagata, Kenji Koide, Yuki Hayakawa, Ryo Suzuki, Kazuma Ikeda, Ozora Sako, Qi Alfred Chen, Takami Sato, Kentaro Yoshioka

Abstract: Accurate localization is essential for enabling modern full self-driving services. These services heavily rely on map-based traffic information to reduce uncertainties in recognizing lane shapes, traffic light locations, and traffic signs. Achieving this level of reliance on map information requires centimeter-level localization accuracy, which is currently only achievable with LiDAR sensors. Howe… ▽ More Accurate localization is essential for enabling modern full self-driving services. These services heavily rely on map-based traffic information to reduce uncertainties in recognizing lane shapes, traffic light locations, and traffic signs. Achieving this level of reliance on map information requires centimeter-level localization accuracy, which is currently only achievable with LiDAR sensors. However, LiDAR is known to be vulnerable to spoofing attacks that emit malicious lasers against LiDAR to overwrite its measurements. Once localization is compromised, the attack could lead the victim off roads or make them ignore traffic lights. Motivated by these serious safety implications, we design SLAMSpoof, the first practical LiDAR spoofing attack on localization systems for self-driving to assess the actual attack significance on autonomous vehicles. SLAMSpoof can effectively find the effective attack location based on our scan matching vulnerability score (SMVS), a point-wise metric representing the potential vulnerability to spoofing attacks. To evaluate the effectiveness of the attack, we conduct real-world experiments on ground vehicles and confirm its high capability in real-world scenarios, inducing position errors of $\geq$4.2 meters (more than typical lane width) for all 3 popular LiDAR-based localization algorithms. We finally discuss the potential countermeasures of this attack. Code is available at https://github.com/Keio-CSG/slamspoof △ Less

Submitted 19 February, 2025; originally announced February 2025.

Comments: 7pages, 6figures, accepted at IEEE International Conference on Robotics and Automation (ICRA) 2025

Hamiltonian Monte Carlo-Based Near-Optimal MIMO Signal Detection

Authors: Junichiro Hagiwara, Toshihiko Nishimura, Takanori Sato, Yasutaka Ogawa, Takeo Ohgane

Abstract: Multiple-input multiple-output (MIMO) technology is essential for the optimal functioning of next-generation wireless networks; however, enhancing its signal-detection performance for improved spectral efficiency is challenging. Here, we propose an approach that transforms the discrete MIMO detection problem into a continuous problem while leveraging the efficient Hamiltonian Monte Carlo algorithm… ▽ More Multiple-input multiple-output (MIMO) technology is essential for the optimal functioning of next-generation wireless networks; however, enhancing its signal-detection performance for improved spectral efficiency is challenging. Here, we propose an approach that transforms the discrete MIMO detection problem into a continuous problem while leveraging the efficient Hamiltonian Monte Carlo algorithm. For this continuous framework, we employ a mixture of t-distributions as the prior distribution. To improve the performance in the coded case further, we treat the likelihood's temperature parameter as a random variable and address its optimization. This treatment leads to the adoption of a horseshoe density for the likelihood. Theoretical analysis and extensive simulations demonstrate that our method achieves near-optimal detection performance while maintaining polynomial computational complexity. This MIMO detection technique can accelerate the development of 6G mobile communication systems. △ Less

Submitted 8 December, 2024; v1 submitted 3 December, 2024; originally announced December 2024.

Comments: This work has been submitted to the IEEE for possible publication

Friction jointing of distributed rigid capacitors to stretchable liquid metal coil for full-body wireless charging clothing

Authors: Takashi Sato, Shinto Watanabe, Ryo Takahashi, Wakako Yukita, Tomoyuki Yokota, Takao Someya, Yoshihito Kawahara, Eiji Iwase, Junya Kurumida

Abstract: For full-body wireless power transfer (WPT), a liquid metal (LM)-based meandered textile coil has been proposed. Multiple rigid capacitors must be inserted in a long coil for efficiency; however, the conventional adhesive jointing suffers from the fragile connection between a rubber tube filled with LM and the capacitor due to the poor adhesion of the rubbers. This paper presents a friction-based… ▽ More For full-body wireless power transfer (WPT), a liquid metal (LM)-based meandered textile coil has been proposed. Multiple rigid capacitors must be inserted in a long coil for efficiency; however, the conventional adhesive jointing suffers from the fragile connection between a rubber tube filled with LM and the capacitor due to the poor adhesion of the rubbers. This paper presents a friction-based jointing, which covers the capacitor with a rigid capsule to enhance the frictional force between the tube and capsule. By experimentally optimizing the capsule design, the LM coil with capacitors showed 3.1 times higher stretch tolerance (31.8 N) and 3.5 times higher bending tolerance (25.9 N) than the adhesive jointing. Moreover, the WPT garment prototype shows excellent mechanical durability against repeated stretching and washing over 100 times. Our full-body meandered textile coil can enable wireless charging to wearable devices around the body for long-term continuous healthcare monitoring, activity recognition, and AR/VR. △ Less

Submitted 2 December, 2024; originally announced December 2024.

Comments: 4 pages, 8 figures, Proceedings of the 38th IEEE International Conference on Micro Electro Mechanical Systems (IEEE MEMS2025)

Journal ref: IEEE MEMS (2025)

Formalization of Differential Privacy in Isabelle/HOL

Authors: Tetsuya Sato, Yasuhiko Minamide

Abstract: Differential privacy is a statistical definition of privacy that has attracted the interest of both academia and industry. Its formulations are easy to understand, but the differential privacy of databases is complicated to determine. One of the reasons for this is that small changes in database programs can break their differential privacy. Therefore, formal verification of differential privacy h… ▽ More Differential privacy is a statistical definition of privacy that has attracted the interest of both academia and industry. Its formulations are easy to understand, but the differential privacy of databases is complicated to determine. One of the reasons for this is that small changes in database programs can break their differential privacy. Therefore, formal verification of differential privacy has been studied for over a decade. In this paper, we propose an Isabelle/HOL library for formalizing differential privacy in a general setting. To our knowledge, it is the first formalization of differential privacy that supports continuous probability distributions. First, we formalize the standard definition of differential privacy and its basic properties. Second, we formalize the Laplace mechanism and its differential privacy. Finally, we formalize the differential privacy of the report noisy max mechanism. △ Less

Submitted 20 October, 2024; originally announced October 2024.

Comments: Draft version

Revisiting Physical-World Adversarial Attack on Traffic Sign Recognition: A Commercial Systems Perspective

Authors: Ningfei Wang, Shaoyuan Xie, Takami Sato, Yunpeng Luo, Kaidi Xu, Qi Alfred Chen

Abstract: Traffic Sign Recognition (TSR) is crucial for safe and correct driving automation. Recent works revealed a general vulnerability of TSR models to physical-world adversarial attacks, which can be low-cost, highly deployable, and capable of causing severe attack effects such as hiding a critical traffic sign or spoofing a fake one. However, so far existing works generally only considered evaluating… ▽ More Traffic Sign Recognition (TSR) is crucial for safe and correct driving automation. Recent works revealed a general vulnerability of TSR models to physical-world adversarial attacks, which can be low-cost, highly deployable, and capable of causing severe attack effects such as hiding a critical traffic sign or spoofing a fake one. However, so far existing works generally only considered evaluating the attack effects on academic TSR models, leaving the impacts of such attacks on real-world commercial TSR systems largely unclear. In this paper, we conduct the first large-scale measurement of physical-world adversarial attacks against commercial TSR systems. Our testing results reveal that it is possible for existing attack works from academia to have highly reliable (100\%) attack success against certain commercial TSR system functionality, but such attack capabilities are not generalizable, leading to much lower-than-expected attack success rates overall. We find that one potential major factor is a spatial memorization design that commonly exists in today's commercial TSR systems. We design new attack success metrics that can mathematically model the impacts of such design on the TSR system-level attack success, and use them to revisit existing attacks. Through these efforts, we uncover 7 novel observations, some of which directly challenge the observations or claims in prior works due to the introduction of the new metrics. △ Less

Submitted 15 September, 2024; originally announced September 2024.

Comments: Accepted by NDSS 2025

An Implementation of Werewolf Agent That does not Truly Trust LLMs

Authors: Takehiro Sato, Shintaro Ozaki, Daisaku Yokoyama

Abstract: Werewolf is an incomplete information game, which has several challenges when creating a computer agent as a player given the lack of understanding of the situation and individuality of utterance (e.g., computer agents are not capable of characterful utterance or situational lying). We propose a werewolf agent that solves some of those difficulties by combining a Large Language Model (LLM) and a r… ▽ More Werewolf is an incomplete information game, which has several challenges when creating a computer agent as a player given the lack of understanding of the situation and individuality of utterance (e.g., computer agents are not capable of characterful utterance or situational lying). We propose a werewolf agent that solves some of those difficulties by combining a Large Language Model (LLM) and a rule-based algorithm. In particular, our agent uses a rule-based algorithm to select an output either from an LLM or a template prepared beforehand based on the results of analyzing conversation history using an LLM. It allows the agent to refute in specific situations, identify when to end the conversation, and behave with persona. This approach mitigated conversational inconsistencies and facilitated logical utterance as a result. We also conducted a qualitative evaluation, which resulted in our agent being perceived as more human-like compared to an unmodified LLM. The agent is freely available for contributing to advance the research in the field of Werewolf game. △ Less

Submitted 2 September, 2024; originally announced September 2024.

Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systems

Authors: Ruochen Jiao, Shaoyuan Xie, Justin Yue, Takami Sato, Lixu Wang, Yixuan Wang, Qi Alfred Chen, Qi Zhu

Abstract: Large Language Models (LLMs) have shown significant promise in real-world decision-making tasks for embodied artificial intelligence, especially when fine-tuned to leverage their inherent common sense and reasoning abilities while being tailored to specific applications. However, this fine-tuning process introduces considerable safety and security vulnerabilities, especially in safety-critical cyb… ▽ More Large Language Models (LLMs) have shown significant promise in real-world decision-making tasks for embodied artificial intelligence, especially when fine-tuned to leverage their inherent common sense and reasoning abilities while being tailored to specific applications. However, this fine-tuning process introduces considerable safety and security vulnerabilities, especially in safety-critical cyber-physical systems. In this work, we propose the first comprehensive framework for Backdoor Attacks against LLM-based Decision-making systems (BALD) in embodied AI, systematically exploring the attack surfaces and trigger mechanisms. Specifically, we propose three distinct attack mechanisms: word injection, scenario manipulation, and knowledge injection, targeting various components in the LLM-based decision-making pipeline. We perform extensive experiments on representative LLMs (GPT-3.5, LLaMA2, PaLM2) in autonomous driving and home robot tasks, demonstrating the effectiveness and stealthiness of our backdoor triggers across various attack channels, with cases like vehicles accelerating toward obstacles and robots placing knives on beds. Our word and knowledge injection attacks achieve nearly 100% success rate across multiple models and datasets while requiring only limited access to the system. Our scenario manipulation attack yields success rates exceeding 65%, reaching up to 90%, and does not require any runtime system intrusion. We also assess the robustness of these attacks against defenses, revealing their resilience. Our findings highlight critical security vulnerabilities in embodied LLM systems and emphasize the urgent need for safeguarding these systems to mitigate potential risks. △ Less

Submitted 30 April, 2025; v1 submitted 27 May, 2024; originally announced May 2024.

Comments: Accepted paper at ICLR 2025, 31 pages, including main paper, references, and appendix

Invisible Reflections: Leveraging Infrared Laser Reflections to Target Traffic Sign Perception

Authors: Takami Sato, Sri Hrushikesh Varma Bhupathiraju, Michael Clifford, Takeshi Sugawara, Qi Alfred Chen, Sara Rampazzi

Abstract: All vehicles must follow the rules that govern traffic behavior, regardless of whether the vehicles are human-driven or Connected Autonomous Vehicles (CAVs). Road signs indicate locally active rules, such as speed limits and requirements to yield or stop. Recent research has demonstrated attacks, such as adding stickers or projected colored patches to signs, that cause CAV misinterpretation, resul… ▽ More All vehicles must follow the rules that govern traffic behavior, regardless of whether the vehicles are human-driven or Connected Autonomous Vehicles (CAVs). Road signs indicate locally active rules, such as speed limits and requirements to yield or stop. Recent research has demonstrated attacks, such as adding stickers or projected colored patches to signs, that cause CAV misinterpretation, resulting in potential safety issues. Humans can see and potentially defend against these attacks. But humans can not detect what they can not observe. We have developed an effective physical-world attack that leverages the sensitivity of filterless image sensors and the properties of Infrared Laser Reflections (ILRs), which are invisible to humans. The attack is designed to affect CAV cameras and perception, undermining traffic sign recognition by inducing misclassification. In this work, we formulate the threat model and requirements for an ILR-based traffic sign perception attack to succeed. We evaluate the effectiveness of the ILR attack with real-world experiments against two major traffic sign recognition architectures on four IR-sensitive cameras. Our black-box optimization methodology allows the attack to achieve up to a 100% attack success rate in indoor, static scenarios and a >80.5% attack success rate in our outdoor, moving vehicle scenarios. We find the latest state-of-the-art certifiable defense is ineffective against ILR attacks as it mis-certifies >33.5% of cases. To address this, we propose a detection strategy based on the physical properties of IR laser reflections which can detect 96% of ILR attacks. △ Less

Submitted 7 January, 2024; originally announced January 2024.

Comments: The first two authors are co-first. Accepted to NDSS '24

PlaNet-S: Automatic Semantic Segmentation of Placenta

Authors: Shinnosuke Yamamoto, Isso Saito, Eichi Takaya, Ayaka Harigai, Tomomi Sato, Tomoya Kobayashi, Kei Takase, Takuya Ueda

Abstract: [Purpose] To develop a fully automated semantic placenta segmentation model that integrates the U-Net and SegNeXt architectures through ensemble learning. [Methods] A total of 218 pregnant women with suspected placental anomalies who underwent magnetic resonance imaging (MRI) were enrolled, yielding 1090 annotated images for developing a deep learning model for placental segmentation. The images w… ▽ More [Purpose] To develop a fully automated semantic placenta segmentation model that integrates the U-Net and SegNeXt architectures through ensemble learning. [Methods] A total of 218 pregnant women with suspected placental anomalies who underwent magnetic resonance imaging (MRI) were enrolled, yielding 1090 annotated images for developing a deep learning model for placental segmentation. The images were standardized and divided into training and test sets. The performance of PlaNet-S, which integrates U-Net and SegNeXt within an ensemble framework, was assessed using Intersection over Union (IoU) and counting connected components (CCC) against the U-Net model. [Results] PlaNet-S had significantly higher IoU (0.73 +/- 0.13) than that of U-Net (0.78 +/- 0.010) (p<0.01). The CCC for PlaNet-S was significantly higher than that for U-Net (p<0.01), matching the ground truth in 86.0\% and 56.7\% of the cases, respectively. [Conclusion]PlaNet-S performed better than the traditional U-Net in placental segmentation tasks. This model addresses the challenges of time-consuming physician-assisted manual segmentation and offers the potential for diverse applications in placental imaging analyses. △ Less

Submitted 26 June, 2024; v1 submitted 18 December, 2023; originally announced December 2023.

Comments: 11 pages, 5 figures, Shinnosuke Yamamoto and Isso Saito equally contributed to this work. In the original submission, there was a typographical error in the reported standard deviation for the Intersection over Union (IoU) values of the PlaNet-S model. The standard deviation was incorrectly listed as 0.01 instead of the correct value of 0.1. This has been corrected in the revised version. J Digit Imaging. Inform. med. (2025)

PHALM: Building a Knowledge Graph from Scratch by Prompting Humans and a Language Model

Authors: Tatsuya Ide, Eiki Murata, Daisuke Kawahara, Takato Yamazaki, Shengzhe Li, Kenta Shinzato, Toshinori Sato

Abstract: Despite the remarkable progress in natural language understanding with pretrained Transformers, neural language models often do not handle commonsense knowledge well. Toward commonsense-aware models, there have been attempts to obtain knowledge, ranging from automatic acquisition to crowdsourcing. However, it is difficult to obtain a high-quality knowledge base at a low cost, especially from scrat… ▽ More Despite the remarkable progress in natural language understanding with pretrained Transformers, neural language models often do not handle commonsense knowledge well. Toward commonsense-aware models, there have been attempts to obtain knowledge, ranging from automatic acquisition to crowdsourcing. However, it is difficult to obtain a high-quality knowledge base at a low cost, especially from scratch. In this paper, we propose PHALM, a method of building a knowledge graph from scratch, by prompting both crowdworkers and a large language model (LLM). We used this method to build a Japanese event knowledge graph and trained Japanese commonsense generation models. Experimental results revealed the acceptability of the built graph and inferences generated by the trained models. We also report the difference in prompting humans and an LLM. Our code, data, and models are available at github.com/nlp-waseda/comet-atomic-ja. △ Less

Submitted 10 October, 2023; originally announced October 2023.

Intriguing Properties of Diffusion Models: An Empirical Study of the Natural Attack Capability in Text-to-Image Generative Models

Authors: Takami Sato, Justin Yue, Nanze Chen, Ningfei Wang, Qi Alfred Chen

Abstract: Denoising probabilistic diffusion models have shown breakthrough performance to generate more photo-realistic images or human-level illustrations than the prior models such as GANs. This high image-generation capability has stimulated the creation of many downstream applications in various areas. However, we find that this technology is actually a double-edged sword: We identify a new type of atta… ▽ More Denoising probabilistic diffusion models have shown breakthrough performance to generate more photo-realistic images or human-level illustrations than the prior models such as GANs. This high image-generation capability has stimulated the creation of many downstream applications in various areas. However, we find that this technology is actually a double-edged sword: We identify a new type of attack, called the Natural Denoising Diffusion (NDD) attack based on the finding that state-of-the-art deep neural network (DNN) models still hold their prediction even if we intentionally remove their robust features, which are essential to the human visual system (HVS), through text prompts. The NDD attack shows a significantly high capability to generate low-cost, model-agnostic, and transferable adversarial attacks by exploiting the natural attack capability in diffusion models. To systematically evaluate the risk of the NDD attack, we perform a large-scale empirical study with our newly created dataset, the Natural Denoising Diffusion Attack (NDDA) dataset. We evaluate the natural attack capability by answering 6 research questions. Through a user study, we find that it can achieve an 88% detection rate while being stealthy to 93% of human subjects; we also find that the non-robust features embedded by diffusion models contribute to the natural attack capability. To confirm the model-agnostic and transferable attack capability, we perform the NDD attack against the Tesla Model 3 and find that 73% of the physically printed attacks can be detected as stop signs. Our hope is that the study and dataset can help our community be aware of the risks in diffusion models and facilitate further research toward robust DNN models. △ Less

Submitted 1 May, 2024; v1 submitted 29 August, 2023; originally announced August 2023.

Does Physical Adversarial Example Really Matter to Autonomous Driving? Towards System-Level Effect of Adversarial Object Evasion Attack

Authors: Ningfei Wang, Yunpeng Luo, Takami Sato, Kaidi Xu, Qi Alfred Chen

Abstract: In autonomous driving (AD), accurate perception is indispensable to achieving safe and secure driving. Due to its safety-criticality, the security of AD perception has been widely studied. Among different attacks on AD perception, the physical adversarial object evasion attacks are especially severe. However, we find that all existing literature only evaluates their attack effect at the targeted A… ▽ More In autonomous driving (AD), accurate perception is indispensable to achieving safe and secure driving. Due to its safety-criticality, the security of AD perception has been widely studied. Among different attacks on AD perception, the physical adversarial object evasion attacks are especially severe. However, we find that all existing literature only evaluates their attack effect at the targeted AI component level but not at the system level, i.e., with the entire system semantics and context such as the full AD pipeline. Thereby, this raises a critical research question: can these existing researches effectively achieve system-level attack effects (e.g., traffic rule violations) in the real-world AD context? In this work, we conduct the first measurement study on whether and how effectively the existing designs can lead to system-level effects, especially for the STOP sign-evasion attacks due to their popularity and severity. Our evaluation results show that all the representative prior works cannot achieve any system-level effects. We observe two design limitations in the prior works: 1) physical model-inconsistent object size distribution in pixel sampling and 2) lack of vehicle plant model and AD system model consideration. Then, we propose SysAdv, a novel system-driven attack design in the AD context and our evaluation results show that the system-level effects can be significantly improved, i.e., the violation rate increases by around 70%. △ Less

Submitted 22 August, 2023; originally announced August 2023.

Comments: Accepted by ICCV 2023

Dialogue Systems Can Generate Appropriate Responses without the Use of Question Marks? -- Investigation of the Effects of Question Marks on Dialogue Systems

Authors: Tomoya Mizumoto, Takato Yamazaki, Katsumasa Yoshikawa, Masaya Ohagi, Toshiki Kawamoto, Toshinori Sato

Abstract: When individuals engage in spoken discourse, various phenomena can be observed that differ from those that are apparent in text-based conversation. While written communication commonly uses a question mark to denote a query, in spoken discourse, queries are frequently indicated by a rising intonation at the end of a sentence. However, numerous speech recognition engines do not append a question ma… ▽ More When individuals engage in spoken discourse, various phenomena can be observed that differ from those that are apparent in text-based conversation. While written communication commonly uses a question mark to denote a query, in spoken discourse, queries are frequently indicated by a rising intonation at the end of a sentence. However, numerous speech recognition engines do not append a question mark to recognized queries, presenting a challenge when creating a spoken dialogue system. Specifically, the absence of a question mark at the end of a sentence can impede the generation of appropriate responses to queries in spoken dialogue systems. Hence, we investigate the impact of question marks on dialogue systems, with the results showing that they have a significant impact. Moreover, we analyze specific examples in an effort to determine which types of utterances have the impact on dialogue systems. △ Less

Submitted 7 August, 2023; originally announced August 2023.

Modular DFR: Digital Delayed Feedback Reservoir Model for Enhancing Design Flexibility

Authors: Sosei Ikeda, Hiromitsu Awano, Takashi Sato

Abstract: A delayed feedback reservoir (DFR) is a type of reservoir computing system well-suited for hardware implementations owing to its simple structure. Most existing DFR implementations use analog circuits that require both digital-to-analog and analog-to-digital converters for interfacing. However, digital DFRs emulate analog nonlinear components in the digital domain, resulting in a lack of design fl… ▽ More A delayed feedback reservoir (DFR) is a type of reservoir computing system well-suited for hardware implementations owing to its simple structure. Most existing DFR implementations use analog circuits that require both digital-to-analog and analog-to-digital converters for interfacing. However, digital DFRs emulate analog nonlinear components in the digital domain, resulting in a lack of design flexibility and higher power consumption. In this paper, we propose a novel modular DFR model that is suitable for fully digital implementations. The proposed model reduces the number of hyperparameters and allows flexibility in the selection of the nonlinear function, which improves the accuracy while reducing the power consumption. We further present two DFR realizations with different nonlinear functions, achieving 10x power reduction and 5.3x throughput improvement while maintaining equal or better accuracy. △ Less

Submitted 5 July, 2023; originally announced July 2023.

Comments: 20 pages, 11 figures. Accepted for publication in the International Conference on Compilers, Architectures, and Synthesis for Embedded Systems (CASES) 2023. Will appear in ACM Transactions on Embedded Computing Systems (TECS)

Towards end-to-end ASP computation

Authors: Taisuke Sato, Akihiro Takemura, Katsumi Inoue

Abstract: We propose an end-to-end approach for answer set programming (ASP) and linear algebraically compute stable models satisfying given constraints. The idea is to implement Lin-Zhao's theorem \cite{Lin04} together with constraints directly in vector spaces as numerical minimization of a cost function constructed from a matricized normal logic program, loop formulas in Lin-Zhao's theorem and constraint… ▽ More We propose an end-to-end approach for answer set programming (ASP) and linear algebraically compute stable models satisfying given constraints. The idea is to implement Lin-Zhao's theorem \cite{Lin04} together with constraints directly in vector spaces as numerical minimization of a cost function constructed from a matricized normal logic program, loop formulas in Lin-Zhao's theorem and constraints, thereby no use of symbolic ASP or SAT solvers involved in our approach. We also propose precomputation that shrinks the program size and heuristics for loop formulas to reduce computational difficulty. We empirically test our approach with programming examples including the 3-coloring and Hamiltonian cycle problems. As our approach is purely numerical and only contains vector/matrix operations, acceleration by parallel technologies such as many-cores and GPUs is expected. △ Less

Submitted 13 June, 2023; v1 submitted 11 June, 2023; originally announced June 2023.

Comments: 29 pages, 9 figures

ACM Class: I.2.4

LiDAR Spoofing Meets the New-Gen: Capability Improvements, Broken Assumptions, and New Attack Strategies

Authors: Takami Sato, Yuki Hayakawa, Ryo Suzuki, Yohsuke Shiiki, Kentaro Yoshioka, Qi Alfred Chen

Abstract: LiDAR (Light Detection And Ranging) is an indispensable sensor for precise long- and wide-range 3D sensing, which directly benefited the recent rapid deployment of autonomous driving (AD). Meanwhile, such a safety-critical application strongly motivates its security research. A recent line of research finds that one can manipulate the LiDAR point cloud and fool object detectors by firing malicious… ▽ More LiDAR (Light Detection And Ranging) is an indispensable sensor for precise long- and wide-range 3D sensing, which directly benefited the recent rapid deployment of autonomous driving (AD). Meanwhile, such a safety-critical application strongly motivates its security research. A recent line of research finds that one can manipulate the LiDAR point cloud and fool object detectors by firing malicious lasers against LiDAR. However, these efforts face 3 critical research gaps: (1) considering only one specific LiDAR (VLP-16); (2) assuming unvalidated attack capabilities; and (3) evaluating object detectors with limited spoofing capability modeling and setup diversity. To fill these critical research gaps, we conduct the first large-scale measurement study on LiDAR spoofing attack capabilities on object detectors with 9 popular LiDARs, covering both first- and new-generation LiDARs, and 3 major types of object detectors trained on 5 different datasets. To facilitate the measurements, we (1) identify spoofer improvements that significantly improve the latest spoofing capability, (2) identify a new object removal attack that overcomes the applicability limitation of the latest method to new-generation LiDARs, and (3) perform novel mathematical modeling for both object injection and removal attacks based on our measurement results. Through this study, we are able to uncover a total of 15 novel findings, including not only completely new ones due to the measurement angle novelty, but also many that can directly challenge the latest understandings in this problem space. We also discuss defenses. △ Less

Submitted 7 February, 2024; v1 submitted 18 March, 2023; originally announced March 2023.

Comments: The first 3 authors are co-first

Journal ref: Network and Distributed System Security (NDSS) Symposium 2024

Learning Representation for Anomaly Detection of Vehicle Trajectories

Authors: Ruochen Jiao, Juyang Bai, Xiangguo Liu, Takami Sato, Xiaowei Yuan, Qi Alfred Chen, Qi Zhu

Abstract: Predicting the future trajectories of surrounding vehicles based on their history trajectories is a critical task in autonomous driving. However, when small crafted perturbations are introduced to those history trajectories, the resulting anomalous (or adversarial) trajectories can significantly mislead the future trajectory prediction module of the ego vehicle, which may result in unsafe planning… ▽ More Predicting the future trajectories of surrounding vehicles based on their history trajectories is a critical task in autonomous driving. However, when small crafted perturbations are introduced to those history trajectories, the resulting anomalous (or adversarial) trajectories can significantly mislead the future trajectory prediction module of the ego vehicle, which may result in unsafe planning and even fatal accidents. Therefore, it is of great importance to detect such anomalous trajectories of the surrounding vehicles for system safety, but few works have addressed this issue. In this work, we propose two novel methods for learning effective and efficient representations for online anomaly detection of vehicle trajectories. Different from general time-series anomaly detection, anomalous vehicle trajectory detection deals with much richer contexts on the road and fewer observable patterns on the anomalous trajectories themselves. To address these challenges, our methods exploit contrastive learning techniques and trajectory semantics to capture the patterns underlying the driving scenarios for effective anomaly detection under supervised and unsupervised settings, respectively. We conduct extensive experiments to demonstrate that our supervised method based on contrastive learning and unsupervised method based on reconstruction with semantic latent space can significantly improve the performance of anomalous trajectory detection in their corresponding settings over various baseline methods. We also demonstrate our methods' generalization ability to detect unseen patterns of anomalies. △ Less

Submitted 8 March, 2023; originally announced March 2023.

Comments: 8 pages paper, in anomaly detection of vehicle trajectory

Near-optimal stochastic MIMO signal detection with a mixture of t-distribution prior

Authors: Junichiro Hagiwara, Kazushi Matsumura, Hiroki Asumi, Yukiko Kasuga, Toshihiko Nishimura, Takanori Sato, Yasutaka Ogawa, Takeo Ohgane

Abstract: Multiple-input multiple-output (MIMO) systems will play a crucial role in future wireless communication, but improving their signal detection performance to increase transmission efficiency remains a challenge. To address this issue, we propose extending the discrete signal detection problem in MIMO systems to a continuous one and applying the Hamiltonian Monte Carlo method, an efficient Markov ch… ▽ More Multiple-input multiple-output (MIMO) systems will play a crucial role in future wireless communication, but improving their signal detection performance to increase transmission efficiency remains a challenge. To address this issue, we propose extending the discrete signal detection problem in MIMO systems to a continuous one and applying the Hamiltonian Monte Carlo method, an efficient Markov chain Monte Carlo algorithm. In our previous studies, we have used a mixture of normal distributions for the prior distribution. In this study, we propose using a mixture of t-distributions, which further improves detection performance. Based on our theoretical analysis and computer simulations, the proposed method can achieve near-optimal signal detection with polynomial computational complexity. This high-performance and practical MIMO signal detection could contribute to the development of the 6th-generation mobile network. △ Less

Submitted 7 March, 2024; v1 submitted 9 January, 2023; originally announced January 2023.

Comments: Published in the 2023 IEEE Global Communications Conference (GLOBECOM)

Formalizing Statistical Causality via Modal Logic

Authors: Yusuke Kawamoto, Tetsuya Sato, Kohei Suenaga

Abstract: We propose a formal language for describing and explaining statistical causality. Concretely, we define Statistical Causality Language (StaCL) for expressing causal effects and specifying the requirements for causal inference. StaCL incorporates modal operators for interventions to express causal properties between probability distributions in different possible worlds in a Kripke model. We formal… ▽ More We propose a formal language for describing and explaining statistical causality. Concretely, we define Statistical Causality Language (StaCL) for expressing causal effects and specifying the requirements for causal inference. StaCL incorporates modal operators for interventions to express causal properties between probability distributions in different possible worlds in a Kripke model. We formalize axioms for probability distributions, interventions, and causal predicates using StaCL formulas. These axioms are expressive enough to derive the rules of Pearl's do-calculus. Finally, we demonstrate by examples that StaCL can be used to specify and explain the correctness of statistical causal inference. △ Less

Submitted 17 September, 2023; v1 submitted 30 October, 2022; originally announced October 2022.

Comments: Full version for the paper accepted at JELIA 2023 (the 18th European Conference on Logics in Artificial Intelligence)

Journal ref: Proc. of the 18th European Conference on Logics in Artificial Intelligence (JELIA 2023), Lecture Notes in Computer Science, Vol.14281, pp.681-696, 2023

Tourist Guidance Robot Based on HyperCLOVA

Authors: Takato Yamazaki, Katsumasa Yoshikawa, Toshiki Kawamoto, Masaya Ohagi, Tomoya Mizumoto, Shuta Ichimura, Yusuke Kida, Toshinori Sato

Abstract: This paper describes our system submitted to Dialogue Robot Competition 2022. Our proposed system is a combined model of rule-based and generation-based dialog systems. The system utilizes HyperCLOVA, a Japanese foundation model, not only to generate responses but also summarization, search information, etc. We also used our original speech recognition system, which was fine-tuned for this dialog… ▽ More This paper describes our system submitted to Dialogue Robot Competition 2022. Our proposed system is a combined model of rule-based and generation-based dialog systems. The system utilizes HyperCLOVA, a Japanese foundation model, not only to generate responses but also summarization, search information, etc. We also used our original speech recognition system, which was fine-tuned for this dialog task. As a result, our system ranked second in the preliminary round and moved on to the finals. △ Less

Submitted 19 October, 2022; originally announced October 2022.

Comments: This paper is part of the proceedings of the Dialogue Robot Competition 2022

Feature-based model selection for object detection from point cloud data

Authors: Kairi Tokuda, Ryoichi Shinkuma, Takehiro Sato, Eiji Oki

Abstract: Smart monitoring using three-dimensional (3D) image sensors has been attracting attention in the context of smart cities. In smart monitoring, object detection from point cloud data acquired by 3D image sensors is implemented for detecting moving objects such as vehicles and pedestrians to ensure safety on the road. However, the features of point cloud data are diversified due to the characteristi… ▽ More Smart monitoring using three-dimensional (3D) image sensors has been attracting attention in the context of smart cities. In smart monitoring, object detection from point cloud data acquired by 3D image sensors is implemented for detecting moving objects such as vehicles and pedestrians to ensure safety on the road. However, the features of point cloud data are diversified due to the characteristics of light detection and ranging (LIDAR) units used as 3D image sensors or the install position of the 3D image sensors. Although a variety of deep learning (DL) models for object detection from point cloud data have been studied to date, no research has considered how to use multiple DL models in accordance with the features of the point cloud data. In this work, we propose a feature-based model selection framework that creates various DL models by using multiple DL methods and by utilizing training data with pseudo incompleteness generated by two artificial techniques: sampling and noise adding. It selects the most suitable DL model for the object detection task in accordance with the features of the point cloud data acquired in the real environment. To demonstrate the effectiveness of the proposed framework, we compare the performance of multiple DL models using benchmark datasets created from the KITTI dataset and present example results of object detection obtained through a real outdoor experiment. Depending on the situation, the detection accuracy varies up to 32% between DL models, which confirms the importance of selecting an appropriate DL model according to the situation. △ Less

Submitted 26 September, 2022; originally announced September 2022.

Comments: Submitted to IEICE Transactions on Communications

Sound and Relatively Complete Belief Hoare Logic for Statistical Hypothesis Testing Programs

Authors: Yusuke Kawamoto, Tetsuya Sato, Kohei Suenaga

Abstract: We propose a new approach to formally describing the requirement for statistical inference and checking whether a program uses the statistical method appropriately. Specifically, we define belief Hoare logic (BHL) for formalizing and reasoning about the statistical beliefs acquired via hypothesis testing. This program logic is sound and relatively complete with respect to a Kripke model for hypoth… ▽ More We propose a new approach to formally describing the requirement for statistical inference and checking whether a program uses the statistical method appropriately. Specifically, we define belief Hoare logic (BHL) for formalizing and reasoning about the statistical beliefs acquired via hypothesis testing. This program logic is sound and relatively complete with respect to a Kripke model for hypothesis tests. We demonstrate by examples that BHL is useful for reasoning about practical issues in hypothesis testing. In our framework, we clarify the importance of prior beliefs in acquiring statistical beliefs through hypothesis testing, and discuss the whole picture of the justification of statistical inference inside and outside the program logic. △ Less

Submitted 8 November, 2023; v1 submitted 15 August, 2022; originally announced August 2022.

Comments: Accepted to the journal Artificial Intelligence (AI); an extended version of the KR'21 conference paper https://proceedings.kr.org/2021/39/

Journal ref: Artificial Intelligence, Vol.326, 104045, Elsevier, 2024

Balancing Cost and Quality: An Exploration of Human-in-the-loop Frameworks for Automated Short Answer Scoring

Authors: Hiroaki Funayama, Tasuku Sato, Yuichiroh Matsubayashi, Tomoya Mizumoto, Jun Suzuki, Kentaro Inui

Abstract: Short answer scoring (SAS) is the task of grading short text written by a learner. In recent years, deep-learning-based approaches have substantially improved the performance of SAS models, but how to guarantee high-quality predictions still remains a critical issue when applying such models to the education field. Towards guaranteeing high-quality predictions, we present the first study of explor… ▽ More Short answer scoring (SAS) is the task of grading short text written by a learner. In recent years, deep-learning-based approaches have substantially improved the performance of SAS models, but how to guarantee high-quality predictions still remains a critical issue when applying such models to the education field. Towards guaranteeing high-quality predictions, we present the first study of exploring the use of human-in-the-loop framework for minimizing the grading cost while guaranteeing the grading quality by allowing a SAS model to share the grading task with a human grader. Specifically, by introducing a confidence estimation method for indicating the reliability of the model predictions, one can guarantee the scoring quality by utilizing only predictions with high reliability for the scoring results and casting predictions with low reliability to human graders. In our experiments, we investigate the feasibility of the proposed framework using multiple confidence estimation methods and multiple SAS datasets. We find that our human-in-the-loop framework allows automatic scoring models and human graders to achieve the target scoring quality. △ Less

Submitted 16 June, 2022; originally announced June 2022.

Comments: 12pages, To be published in proceedings of AIED2022

Divergences on Monads for Relational Program Logics

Authors: Tetsuya Sato, Shin-ya Katsumata

Abstract: Several relational program logics have been introduced for integrating reasoning about relational properties of programs and measurement of quantitative difference between computational effects. Towards a general framework for such logics, in this paper, we formalize quantitative difference between computational effects as divergence on monad, then develop a relational program logic acRL that supp… ▽ More Several relational program logics have been introduced for integrating reasoning about relational properties of programs and measurement of quantitative difference between computational effects. Towards a general framework for such logics, in this paper, we formalize quantitative difference between computational effects as divergence on monad, then develop a relational program logic acRL that supports generic computational effects and divergences on them. To give a categorical semantics of acRL supporting divergences, we give a method to obtain graded strong relational liftings from divergences on monads. We derive two instantiations of acRL for the verification of 1) various differential privacy of higher-order functional probabilistic programs and 2) difference of distribution of costs between higher-order functional programs with probabilistic choice and cost counting operations. △ Less

Submitted 12 June, 2022; originally announced June 2022.

Comments: Preprint

Building a Personalized Dialogue System with Prompt-Tuning

Authors: Tomohito Kasahara, Daisuke Kawahara, Nguyen Tung, Shengzhe Li, Kenta Shinzato, Toshinori Sato

Abstract: Dialogue systems without consistent responses are not fascinating. In this study, we build a dialogue system that can respond based on a given character setting (persona) to bring consistency. Considering the trend of the rapidly increasing scale of language models, we propose an approach that uses prompt-tuning, which has low learning costs, on pre-trained large-scale language models. The results… ▽ More Dialogue systems without consistent responses are not fascinating. In this study, we build a dialogue system that can respond based on a given character setting (persona) to bring consistency. Considering the trend of the rapidly increasing scale of language models, we propose an approach that uses prompt-tuning, which has low learning costs, on pre-trained large-scale language models. The results of automatic and manual evaluations in English and Japanese show that it is possible to build a dialogue system with more natural and personalized responses using less computational resources than fine-tuning. △ Less

Submitted 10 June, 2022; originally announced June 2022.

Comments: Accepted to NAACL 2022 SRW

Semi-supervised Semantics-guided Adversarial Training for Trajectory Prediction

Authors: Ruochen Jiao, Xiangguo Liu, Takami Sato, Qi Alfred Chen, Qi Zhu

Abstract: Predicting the trajectories of surrounding objects is a critical task for self-driving vehicles and many other autonomous systems. Recent works demonstrate that adversarial attacks on trajectory prediction, where small crafted perturbations are introduced to history trajectories, may significantly mislead the prediction of future trajectories and induce unsafe planning. However, few works have add… ▽ More Predicting the trajectories of surrounding objects is a critical task for self-driving vehicles and many other autonomous systems. Recent works demonstrate that adversarial attacks on trajectory prediction, where small crafted perturbations are introduced to history trajectories, may significantly mislead the prediction of future trajectories and induce unsafe planning. However, few works have addressed enhancing the robustness of this important safety-critical task.In this paper, we present a novel adversarial training method for trajectory prediction. Compared with typical adversarial training on image tasks, our work is challenged by more random input with rich context and a lack of class labels. To address these challenges, we propose a method based on a semi-supervised adversarial autoencoder, which models disentangled semantic features with domain knowledge and provides additional latent labels for the adversarial training. Extensive experiments with different types of attacks demonstrate that our Semisupervised Semantics-guided Adversarial Training (SSAT) method can effectively mitigate the impact of adversarial attacks by up to 73% and outperform other popular defense methods. In addition, experiments show that our method can significantly improve the system's robust generalization to unseen patterns of attacks. We believe that such semantics-guided architecture and advancement on robust generalization is an important step for developing robust prediction models and enabling safe decision-making. △ Less

Submitted 20 March, 2023; v1 submitted 27 May, 2022; originally announced May 2022.

Comments: 11 pages, adversarial training for trajectory prediction

Operations Smart Contract to Realize Decentralized System Operations Workflow for Consortium Blockchain

Authors: Tatsuya Sato, Taku Shimosawa, Yosuke Himura

Abstract: Enterprises have paid attention to consortium blockchains like Hyperledger Fabric, which is one of the most promising platforms, for efficient decentralized transactions without depending on any particular organization. A consortium blockchain-based system will be typically built across multiple organizations. In such blockchain-based systems, system operations across multiple organizations in a d… ▽ More Enterprises have paid attention to consortium blockchains like Hyperledger Fabric, which is one of the most promising platforms, for efficient decentralized transactions without depending on any particular organization. A consortium blockchain-based system will be typically built across multiple organizations. In such blockchain-based systems, system operations across multiple organizations in a decentralized manner are essential to maintain the value of introducing consortium blockchains. Decentralized system operations have recently been becoming realistic with the evolution of consortium blockchains. For instance, the release of Hyperledger Fabric v2.x, in which individual operational tasks for a blockchain network, such as command execution of configuration change of channels (Fabric's sub-networks) and upgrade of chaincodes (Fabric's smart contracts), can be partially executed in a decentralized manner. However, the operations workflows also include the preceding procedure of pre-sharing, coordinating, and pre-agreeing the operational information (e.g., configuration parameters) among organizations, after which operation executions can be conducted, and this preceding procedure relies on costly manual tasks. To realize efficient decentralized operations workflows for consortium blockchain-based systems in general, we propose a decentralized inter-organizational operations method that we call Operations Smart Contract (OpsSC), which defines an operations workflow as a smart contract. Furthermore, we design and implement OpsSC for blockchain network operations with Hyperledger Fabric v2.x. This paper presents OpsSC for operating channels and chaincodes, which are essential for managing the blockchain networks, through clarifying detailed workflows of those operations. The implementation of OpsSC has been open-sourced and registered as one of Hyperledger Labs projects. △ Less

Submitted 10 May, 2022; originally announced May 2022.

Journal ref: IEICE Transactions on Communications, 2022

A Comparative Study of Meter Detection Methods for Automated Infrastructure Inspection

Authors: Yusuke Ohtsubo, Takuto Sato, Hirohiko Sagawa

Abstract: In order to read meter values from a camera on an autonomous inspection robot with positional errors, it is necessary to detect meter regions from the image. In this study, we developed shape-based, texture-based, and background information-based methods as meter area detection techniques and compared their effectiveness for meters of different shapes and sizes. As a result, we confirmed that the… ▽ More In order to read meter values from a camera on an autonomous inspection robot with positional errors, it is necessary to detect meter regions from the image. In this study, we developed shape-based, texture-based, and background information-based methods as meter area detection techniques and compared their effectiveness for meters of different shapes and sizes. As a result, we confirmed that the background information-based method can detect the farthest meters regardless of the shape and number of meters, and can stably detect meters with a diameter of 40px. △ Less

Submitted 24 April, 2022; originally announced April 2022.

Comments: 2 pages, in Japanese language

Towards Driving-Oriented Metric for Lane Detection Models

Authors: Takami Sato, Qi Alfred Chen

Abstract: After the 2017 TuSimple Lane Detection Challenge, its dataset and evaluation based on accuracy and F1 score have become the de facto standard to measure the performance of lane detection methods. While they have played a major role in improving the performance of lane detection methods, the validity of this evaluation method in downstream tasks has not been adequately researched. In this study, we… ▽ More After the 2017 TuSimple Lane Detection Challenge, its dataset and evaluation based on accuracy and F1 score have become the de facto standard to measure the performance of lane detection methods. While they have played a major role in improving the performance of lane detection methods, the validity of this evaluation method in downstream tasks has not been adequately researched. In this study, we design 2 new driving-oriented metrics for lane detection: End-to-End Lateral Deviation metric (E2E-LD) is directly formulated based on the requirements of autonomous driving, a core downstream task of lane detection; Per-frame Simulated Lateral Deviation metric (PSLD) is a lightweight surrogate metric of E2E-LD. To evaluate the validity of the metrics, we conduct a large-scale empirical study with 4 major types of lane detection approaches on the TuSimple dataset and our newly constructed dataset Comma2k19-LD. Our results show that the conventional metrics have strongly negative correlations ($\leq$-0.55) with E2E-LD, meaning that some recent improvements purely targeting the conventional metrics may not have led to meaningful improvements in autonomous driving, but rather may actually have made it worse by overfitting to the conventional metrics. As autonomous driving is a security/safety-critical system, the underestimation of robustness hinders the sound development of practical lane detection models. We hope that our study will help the community achieve more downstream task-aware evaluations for lane detection. △ Less

Submitted 31 March, 2022; originally announced March 2022.

Comments: Accepted to CVPR 2022. arXiv admin note: text overlap with arXiv:2107.02488

SoK: On the Semantic AI Security in Autonomous Driving

Authors: Junjie Shen, Ningfei Wang, Ziwen Wan, Yunpeng Luo, Takami Sato, Zhisheng Hu, Xinyang Zhang, Shengjian Guo, Zhenyu Zhong, Kang Li, Ziming Zhao, Chunming Qiao, Qi Alfred Chen

Abstract: Autonomous Driving (AD) systems rely on AI components to make safety and correct driving decisions. Unfortunately, today's AI algorithms are known to be generally vulnerable to adversarial attacks. However, for such AI component-level vulnerabilities to be semantically impactful at the system level, it needs to address non-trivial semantic gaps both (1) from the system-level attack input spaces to… ▽ More Autonomous Driving (AD) systems rely on AI components to make safety and correct driving decisions. Unfortunately, today's AI algorithms are known to be generally vulnerable to adversarial attacks. However, for such AI component-level vulnerabilities to be semantically impactful at the system level, it needs to address non-trivial semantic gaps both (1) from the system-level attack input spaces to those at AI component level, and (2) from AI component-level attack impacts to those at the system level. In this paper, we define such research space as semantic AI security as opposed to generic AI security. Over the past 5 years, increasingly more research works are performed to tackle such semantic AI security challenges in AD context, which has started to show an exponential growth trend. In this paper, we perform the first systematization of knowledge of such growing semantic AD AI security research space. In total, we collect and analyze 53 such papers, and systematically taxonomize them based on research aspects critical for the security field. We summarize 6 most substantial scientific gaps observed based on quantitative comparisons both vertically among existing AD AI security works and horizontally with security works from closely-related domains. With these, we are able to provide insights and potential future directions not only at the design level, but also at the research goal, methodology, and community levels. To address the most critical scientific methodology-level gap, we take the initiative to develop an open-source, uniform, and extensible system-driven evaluation platform, named PASS, for the semantic AD AI security research community. We also use our implemented platform prototype to showcase the capabilities and benefits of such a platform using representative semantic AD AI attacks. △ Less

Submitted 26 April, 2024; v1 submitted 10 March, 2022; originally announced March 2022.

Comments: Project website: https://sites.google.com/view/cav-sec/pass

Integrated ICN and CDN Slice as a Service

Authors: Ilias Benkacem, M. Bagaa, T. Taleb, Q. N. Nguyen, T. Tsuda, T. Sato

Abstract: In this article, we leverage Network Function Virtualization (NFV) and Multi-Access Edge Computing (MEC) technologies, proposing a system which integrates ICN (Information-Centric Network) with CDN (Content Delivery Network) to provide an efficient content delivery service. The proposed system combines the dynamic CDN slicing concept with the NDN (Named Data Network) based ICN slicing concept to a… ▽ More In this article, we leverage Network Function Virtualization (NFV) and Multi-Access Edge Computing (MEC) technologies, proposing a system which integrates ICN (Information-Centric Network) with CDN (Content Delivery Network) to provide an efficient content delivery service. The proposed system combines the dynamic CDN slicing concept with the NDN (Named Data Network) based ICN slicing concept to avoid core network congestion. A dynamic CDN slice is deployed to cache content at optimal locations depending on the nature of the content and the geographical distributions of potential viewers. Virtual cache servers, along with supporting virtual transcoders, are placed across a cloud belonging to multiple-administrative domains, forming a CDN slice. The ICN slice is, in turn, used for the regional distribution of content, leveraging the name-based access and the autonomic in-network content caching. This enables the delivery of content from nearby network nodes, avoiding the duplicate transfer of content and also ensuring shorter response times. Our experiments demonstrate that integrated ICN/CDN is better than traditional CDN in almost all aspects, including service scalability, reliability, and quality of service. △ Less

Submitted 3 January, 2022; originally announced January 2022.

MatSat: a matrix-based differentiable SAT solver

Authors: Taisuke Sato, Ryosuke Kojima

Abstract: We propose a new approach to SAT solving which solves SAT problems in vector spaces as a cost minimization problem of a non-negative differentiable cost function J^sat. In our approach, a solution, i.e., satisfying assignment, for a SAT problem in n variables is represented by a binary vector u in {0,1}^n that makes J^sat(u) zero. We search for such u in a vector space R^n by cost minimization, i.… ▽ More We propose a new approach to SAT solving which solves SAT problems in vector spaces as a cost minimization problem of a non-negative differentiable cost function J^sat. In our approach, a solution, i.e., satisfying assignment, for a SAT problem in n variables is represented by a binary vector u in {0,1}^n that makes J^sat(u) zero. We search for such u in a vector space R^n by cost minimization, i.e., starting from an initial u_0 and minimizing J to zero while iteratively updating u by Newton's method. We implemented our approach as a matrix-based differential SAT solver MatSat. Although existing main-stream SAT solvers decide each bit of a solution assignment one by one, be they of conflict driven clause learning (CDCL) type or of stochastic local search (SLS) type, MatSat fundamentally differs from them in that it continuously approach a solution in a vector space. We conducted an experiment to measure the scalability of MatSat with random 3-SAT problems in which MatSat could find a solution up to n=10^5 variables. We also compared MatSat with four state-of-the-art SAT solvers including winners of SAT competition 2018 and SAT Race 2019 in terms of time for finding a solution, using a random benchmark set from SAT 2018 competition and an artificial random 3-SAT instance set. The result shows that MatSat comes in second in both test sets and outperforms all the CDCL type solvers. △ Less

Submitted 14 August, 2021; originally announced August 2021.

Meeting the SDGs : Enabling the Goals by Cooperation with Crowd using a Conversational AI Platform

Authors: J. Haqbeen, T. Ito, S. Sahab, R. Hadfi, T. Sato, S. Okuhara

Abstract: In this paper, we report about a large-scale online discussion with 1099 citizens on the Afghanistan Sustainable Development Goals. In this paper, we report about a large-scale online discussion with 1099 citizens on the Afghanistan Sustainable Development Goals. △ Less

Submitted 9 June, 2021; originally announced July 2021.

Comments: 7 pages, 6 figures, 1 table, To appear as a conference paper at KICSS 2020

On Robustness of Lane Detection Models to Physical-World Adversarial Attacks in Autonomous Driving

Authors: Takami Sato, Qi Alfred Chen

Abstract: After the 2017 TuSimple Lane Detection Challenge, its evaluation based on accuracy and F1 score has become the de facto standard to measure the performance of lane detection methods. In this work, we conduct the first large-scale empirical study to evaluate the robustness of state-of-the-art lane detection methods under physical-world adversarial attacks in autonomous driving. We evaluate 4 major… ▽ More After the 2017 TuSimple Lane Detection Challenge, its evaluation based on accuracy and F1 score has become the de facto standard to measure the performance of lane detection methods. In this work, we conduct the first large-scale empirical study to evaluate the robustness of state-of-the-art lane detection methods under physical-world adversarial attacks in autonomous driving. We evaluate 4 major types of lane detection approaches with the conventional evaluation and end-to-end evaluation in autonomous driving scenarios and then discuss the security proprieties of each lane detection model. We demonstrate that the conventional evaluation fails to reflect the robustness in end-to-end autonomous driving scenarios. Our results show that the most robust model on the conventional metrics is the least robust in the end-to-end evaluation. Although the competition dataset and its metrics have played a substantial role in developing performant lane detection methods along with the rapid development of deep neural networks, the conventional evaluation is becoming obsolete and the gap between the metrics and practicality is critical. We hope that our study will help the community make further progress in building a more comprehensive framework to evaluate lane detection models. △ Less

Submitted 6 July, 2021; originally announced July 2021.

Higher-order probabilistic adversarial computations: Categorical semantics and program logics

Authors: Alejandro Aguirre, Gilles Barthe, Marco Gaboardi, Deepak Garg, Shin-ya Katsumata, Tetsuya Sato

Abstract: Adversarial computations are a widely studied class of computations where resource-bounded probabilistic adversaries have access to oracles, i.e., probabilistic procedures with private state. These computations arise routinely in several domains, including security, privacy and machine learning. In this paper, we develop program logics for reasoning about adversarial computations in a higher-order… ▽ More Adversarial computations are a widely studied class of computations where resource-bounded probabilistic adversaries have access to oracles, i.e., probabilistic procedures with private state. These computations arise routinely in several domains, including security, privacy and machine learning. In this paper, we develop program logics for reasoning about adversarial computations in a higher-order setting. Our logics are built on top of a simply typed $λ$-calculus extended with a graded monad for probabilities and state. The grading is used to model and restrict the memory footprint and the cost (in terms of oracle calls) of computations. Under this view, an adversary is a higher-order expression that expects as arguments the code of its oracles. We develop unary program logics for reasoning about error probabilities and expected values, and a relational logic for reasoning about coupling-based properties. All logics feature rules for adversarial computations, and yield guarantees that are valid for all adversaries that satisfy a fixed resource policy. We prove the soundness of the logics in the category of quasi-Borel spaces, using a general notion of graded predicate liftings, and we use logical relations over graded predicate liftings to establish the soundness of proof rules for adversaries. We illustrate the working of our logics with simple but illustrative examples. △ Less

Submitted 2 July, 2021; originally announced July 2021.

Comments: Full version of ICFP 21 paper

Design and Implementation of 5G eHealth Systems, Technologies, Use Cases and Future Challenges

Authors: Di Zhang, Joel J. P. C. Rodrigues, Yunkai Zhai, Takuro Sato

Abstract: Fifth generation (5G) aims to connect massive devices with even higher reliability, lower latency and even faster transmission speed, which are vital for implementing the e-health systems. However, the current efforts on 5G e-health systems are still not enough to accomplish its full blueprint. In this article, we first discuss the related technologies from physical layer, upper layer and cross la… ▽ More Fifth generation (5G) aims to connect massive devices with even higher reliability, lower latency and even faster transmission speed, which are vital for implementing the e-health systems. However, the current efforts on 5G e-health systems are still not enough to accomplish its full blueprint. In this article, we first discuss the related technologies from physical layer, upper layer and cross layer perspectives on designing the 5G e-health systems. We afterwards elaborate two use cases according to our implementations, i.e., 5G e-health systems for remote health and 5G e-health systems for Covid-19 pandemic containment. We finally envision the future research trends and challenges of 5G e-health systems. △ Less

Submitted 10 July, 2021; v1 submitted 9 June, 2021; originally announced June 2021.

Tree Edit Distance with Variables. Measuring the Similarity between Mathematical Formulas

Authors: Tatsuya Akutsu, Tomoya Mori, Naotoshi Nakamura, Satoshi Kozawa, Yuhei Ueno, Thomas N. Sato

Abstract: In this article, we propose tree edit distance with variables, which is an extension of the tree edit distance to handle trees with variables and has a potential application to measuring the similarity between mathematical formulas, especially, those appearing in mathematical models of biological systems. We analyze the computational complexities of several variants of this new model. In particula… ▽ More In this article, we propose tree edit distance with variables, which is an extension of the tree edit distance to handle trees with variables and has a potential application to measuring the similarity between mathematical formulas, especially, those appearing in mathematical models of biological systems. We analyze the computational complexities of several variants of this new model. In particular, we show that the problem is NP-complete for ordered trees. We also show for unordered trees that the problem of deciding whether or not the distance is 0 is graph isomorphism complete but can be solved in polynomial time if the maximum outdegree of input trees is bounded by a constant. This distance model is then extended for measuring the difference/similarity between two systems of differential equations, for which results of preliminary computational experiments using biological models are provided. △ Less

Submitted 11 May, 2021; originally announced May 2021.

Secure Artificial Intelligence of Things for Implicit Group Recommendations

Authors: Keping Yu, Zhiwei Guo, Yu Shen, Wei Wang, Jerry Chun-Wei Lin, Takuro Sato

Abstract: The emergence of Artificial Intelligence of Things (AIoT) has provided novel insights for many social computing applications such as group recommender systems. As distance among people has been greatly shortened, it has been a more general demand to provide personalized services to groups instead of individuals. In order to capture group-level preference features from individuals, existing methods… ▽ More The emergence of Artificial Intelligence of Things (AIoT) has provided novel insights for many social computing applications such as group recommender systems. As distance among people has been greatly shortened, it has been a more general demand to provide personalized services to groups instead of individuals. In order to capture group-level preference features from individuals, existing methods were mostly established via aggregation and face two aspects of challenges: secure data management workflow is absent, and implicit preference feedbacks is ignored. To tackle current difficulties, this paper proposes secure Artificial Intelligence of Things for implicit Group Recommendations (SAIoT-GR). As for hardware module, a secure IoT structure is developed as the bottom support platform. As for software module, collaborative Bayesian network model and non-cooperative game are can be introduced as algorithms. Such a secure AIoT architecture is able to maximize the advantages of the two modules. In addition, a large number of experiments are carried out to evaluate the performance of the SAIoT-GR in terms of efficiency and robustness. △ Less

Submitted 23 April, 2021; originally announced April 2021.

Assessment of image generation by quantum annealer

Authors: Takehito Sato, Masayuki Ohzeki, Kazuyuki Tanaka

Abstract: Quantum annealing was originally proposed as an approach for solving combinatorial optimisation problems using quantum effects. D-Wave Systems has released a production model of quantum annealing hardware. However, the inherent noise and various environmental factors in the hardware hamper the determination of optimal solutions. In addition, the freezing effect in regions with weak quantum fluctua… ▽ More Quantum annealing was originally proposed as an approach for solving combinatorial optimisation problems using quantum effects. D-Wave Systems has released a production model of quantum annealing hardware. However, the inherent noise and various environmental factors in the hardware hamper the determination of optimal solutions. In addition, the freezing effect in regions with weak quantum fluctuations generates outputs approximately following a Gibbs--Boltzmann distribution at an extremely low temperature. Thus, a quantum annealer may also serve as a fast sampler for the Ising spin-glass problem, and several studies have investigated Boltzmann machine learning using a quantum annealer. Previous developments have focused on comparing the performance in the standard distance of the resulting distributions between conventional methods in classical computers and sampling by a quantum annealer. In this study, we focused on the performance of a quantum annealer as a generative model. To evaluate its performance, we prepared a discriminator given by a neural network trained on an a priori dataset. The evaluation results show a higher performance of quantum annealing compared with the classical approach for Boltzmann machine learning. △ Less

Submitted 15 March, 2021; originally announced March 2021.

Comments: 10 pages, 8 figures

End-to-end Uncertainty-based Mitigation of Adversarial Attacks to Automated Lane Centering

Authors: Ruochen Jiao, Hengyi Liang, Takami Sato, Junjie Shen, Qi Alfred Chen, Qi Zhu

Abstract: In the development of advanced driver-assistance systems (ADAS) and autonomous vehicles, machine learning techniques that are based on deep neural networks (DNNs) have been widely used for vehicle perception. These techniques offer significant improvement on average perception accuracy over traditional methods, however, have been shown to be susceptible to adversarial attacks, where small perturba… ▽ More In the development of advanced driver-assistance systems (ADAS) and autonomous vehicles, machine learning techniques that are based on deep neural networks (DNNs) have been widely used for vehicle perception. These techniques offer significant improvement on average perception accuracy over traditional methods, however, have been shown to be susceptible to adversarial attacks, where small perturbations in the input may cause significant errors in the perception results and lead to system failure. Most prior works addressing such adversarial attacks focus only on the sensing and perception modules. In this work, we propose an end-to-end approach that addresses the impact of adversarial attacks throughout perception, planning, and control modules. In particular, we choose a target ADAS application, the automated lane centering system in OpenPilot, quantify the perception uncertainty under adversarial attacks, and design a robust planning and control module accordingly based on the uncertainty analysis. We evaluate our proposed approach using both the public dataset and production-grade autonomous driving simulator. The experiment results demonstrate that our approach can effectively mitigate the impact of adversarial attacks and can achieve 55% to 90% improvement over the original OpenPilot. △ Less

Submitted 27 February, 2021; originally announced March 2021.

Comments: 8 pages for conference

Virtual Secure Platform: A Five-Stage Pipeline Processor over TFHE

Authors: Kotaro Matsuoka, Ryotaro Banno, Naoki Matsumoto, Takashi Sato, Song Bian

Abstract: We present Virtual Secure Platform (VSP), the first comprehensive platform that implements a multi-opcode general-purpose sequential processor over Fully Homomorphic Encryption (FHE) for Secure Multi-Party Computation (SMPC). VSP protects both the data and functions on which the data are evaluated from the adversary in a secure computation offloading situation like cloud computing. We proposed a c… ▽ More We present Virtual Secure Platform (VSP), the first comprehensive platform that implements a multi-opcode general-purpose sequential processor over Fully Homomorphic Encryption (FHE) for Secure Multi-Party Computation (SMPC). VSP protects both the data and functions on which the data are evaluated from the adversary in a secure computation offloading situation like cloud computing. We proposed a complete processor architecture with a five-stage pipeline, which improves the performance of the VSP by providing more parallelism in circuit evaluation. In addition, we also designed a custom Instruction Set Architecture (ISA) to reduce the gate count of our processor, along with an entire set of toolchains to ensure that arbitrary C programs can be compiled into our custom ISA. In order to speed up instruction evaluation over VSP, CMUX Memory based ROM and RAM constructions over FHE are also proposed. Our experiments show that both the pipelined architecture and the CMUX Memory technique are effective in improving the performance of the proposed processor. We provide an open-source implementation of VSP which achieves a per-instruction latency of less than 1 second. We demonstrate that compared to the best existing processor over FHE, our implementation runs nearly 1,600$\times$ faster. △ Less

Submitted 19 October, 2020; originally announced October 2020.

Comments: Accepted in USENIX Security '21. This is a long version

An Anonymous Trust-Marking Scheme on Blockchain Systems

Authors: Teppei Sato, Keita Emura, Tomoki Fujitani, Kazumasa Omote

Abstract: During the Coincheck incident, which recorded the largest damages in cryptocurrency history in 2018, it was demonstrated that using Mosaic token can have a certain effect. Although it seems attractive to employ tokens as countermeasures for cryptocurrency leakage, Mosaic is a specific token for the New Economy Movement (NEM) cryptocurrency and is not employed for other blockchain systems or crypto… ▽ More During the Coincheck incident, which recorded the largest damages in cryptocurrency history in 2018, it was demonstrated that using Mosaic token can have a certain effect. Although it seems attractive to employ tokens as countermeasures for cryptocurrency leakage, Mosaic is a specific token for the New Economy Movement (NEM) cryptocurrency and is not employed for other blockchain systems or cryptocurrencies. Moreover, although some volunteers tracked leaked NEM using Mosaic in the CoinCheck incident, it would be better to verify that the volunteers can be trusted. Simultaneously, if someone (e.g., who stole cryptocurrencies) can identify the volunteers, then that person or organization may be targets of them. In this paper, we propose an anonymous trust-marking scheme on blockchain systems that is universally applicable to any cryptocurrency. In our scheme, entities called token admitters are allowed to generate tokens adding trustworthiness or untrustworthiness to addresses. Anyone can anonymously verify whether these tokens were issued by a token admitter. Simultaneously, only the designated auditor and no one else, including nondesignated auditors, can identify the token admitters. Our scheme is based on accountable ring signatures and commitment, and is implemented on an elliptic curve called Curve25519, and we confirm that both cryptographic tools are efficient. Moreover, we also confirm that our scheme is applicable to Bitcoin, Ethereum, and NEM. △ Less

Submitted 25 February, 2021; v1 submitted 1 October, 2020; originally announced October 2020.

Comments: An extended abstract appeared at the 3rd IEEE International Conference on Blockchain and Cryptocurrency, ICBC 2021. This is the full version

Dirty Road Can Attack: Security of Deep Learning based Automated Lane Centering under Physical-World Attack

Authors: Takami Sato, Junjie Shen, Ningfei Wang, Yunhan Jack Jia, Xue Lin, Qi Alfred Chen

Abstract: Automated Lane Centering (ALC) systems are convenient and widely deployed today, but also highly security and safety critical. In this work, we are the first to systematically study the security of state-of-the-art deep learning based ALC systems in their designed operational domains under physical-world adversarial attacks. We formulate the problem with a safety-critical attack goal, and a novel… ▽ More Automated Lane Centering (ALC) systems are convenient and widely deployed today, but also highly security and safety critical. In this work, we are the first to systematically study the security of state-of-the-art deep learning based ALC systems in their designed operational domains under physical-world adversarial attacks. We formulate the problem with a safety-critical attack goal, and a novel and domain-specific attack vector: dirty road patches. To systematically generate the attack, we adopt an optimization-based approach and overcome domain-specific design challenges such as camera frame inter-dependencies due to attack-influenced vehicle control, and the lack of objective function design for lane detection models. We evaluate our attack on a production ALC using 80 scenarios from real-world driving traces. The results show that our attack is highly effective with over 97.5% success rates and less than 0.903 sec average success time, which is substantially lower than the average driver reaction time. This attack is also found (1) robust to various real-world factors such as lighting conditions and view angles, (2) general to different model designs, and (3) stealthy from the driver's view. To understand the safety impacts, we conduct experiments using software-in-the-loop simulation and attack trace injection in a real vehicle. The results show that our attack can cause a 100% collision rate in different scenarios, including when tested with common safety features such as automatic emergency braking. We also evaluate and discuss defenses. △ Less

Submitted 13 June, 2021; v1 submitted 14 September, 2020; originally announced September 2020.

Comments: Accepted to Usenix Security '21

FedNNNN: Norm-Normalized Neural Network Aggregation for Fast and Accurate Federated Learning

Authors: Kenta Nagura, Song Bian, Takashi Sato

Abstract: Federated learning (FL) is a distributed learning protocol in which a server needs to aggregate a set of models learned some independent clients to proceed the learning process. At present, model averaging, known as FedAvg, is one of the most widely adapted aggregation techniques. However, it is known to yield the models with degraded prediction accuracy and slow convergence. In this work, we find… ▽ More Federated learning (FL) is a distributed learning protocol in which a server needs to aggregate a set of models learned some independent clients to proceed the learning process. At present, model averaging, known as FedAvg, is one of the most widely adapted aggregation techniques. However, it is known to yield the models with degraded prediction accuracy and slow convergence. In this work, we find out that averaging models from different clients significantly diminishes the norm of the update vectors, resulting in slow learning rate and low prediction accuracy. Therefore, we propose a new aggregation method called FedNNNN. Instead of simple model averaging, we adjust the norm of the update vector and introduce momentum control techniques to improve the aggregation effectiveness of FL. As a demonstration, we evaluate FedNNNN on multiple datasets and scenarios with different neural network models, and observe up to 5.4% accuracy improvement. △ Less

Submitted 11 August, 2020; originally announced August 2020.