-
Scene-Segmentation-Based Exposure Compensation for Tone Mapping of High Dynamic Range Scenes
Authors:
Yuma Kinoshita,
Hitoshi Kiya
Abstract:
We propose a novel scene-segmentation-based exposure compensation method for multi-exposure image fusion (MEF) based tone mapping. The aim of MEF-based tone mapping is to display high dynamic range (HDR) images on devices with limited dynamic range. To achieve this, this method generates a stack of differently exposed images from an input HDR image and fuses them into a single image. Our approach…
▽ More
We propose a novel scene-segmentation-based exposure compensation method for multi-exposure image fusion (MEF) based tone mapping. The aim of MEF-based tone mapping is to display high dynamic range (HDR) images on devices with limited dynamic range. To achieve this, this method generates a stack of differently exposed images from an input HDR image and fuses them into a single image. Our approach addresses the limitations of MEF-based tone mapping with existing segmentation-based exposure compensation, which often result in visually unappealing outcomes due to inappropriate exposure value selection. The proposed exposure compensation method first segments the input HDR image into subregions based on luminance values of pixels. It then determines exposure values for multi-exposure images to maximize contrast between regions while preserving relative luminance relationships. This approach contrasts with conventional methods that may invert luminance relationships or compromise contrast between regions. Additionally, we present an improved technique for calculating fusion weights to better reflect the effects of exposure compensation in the final fused image. In a simulation experiment to evaluate the quality of tone-mapped images, the MEF-based tone mapping with the proposed method outperforms three typical tone mapping methods including conventional MEF-based one, in terms of the tone mapped image quality index (TMQI).
△ Less
Submitted 21 October, 2024;
originally announced October 2024.
-
On the Security of Bitstream-level JPEG Encryption with Restart Markers
Authors:
Mare Hirose,
Shoko Imaizumi,
Hitoshi Kiya
Abstract:
This paper aims to evaluate the security of a bitstream-level JPEG encryption method using restart (RST) markers, where encrypted image can keep the JPEG file format with the same file size as non-encrypted image. Data encrypted using this method can be decoded without altering header information by employing a standard JPEG decoder. Moreover, the use of RST markers enables the definition of exten…
▽ More
This paper aims to evaluate the security of a bitstream-level JPEG encryption method using restart (RST) markers, where encrypted image can keep the JPEG file format with the same file size as non-encrypted image. Data encrypted using this method can be decoded without altering header information by employing a standard JPEG decoder. Moreover, the use of RST markers enables the definition of extended blocks divided by the markers, so spatially partial encryption and block-permutation-based encryption can be carried out. However, the security of the method was evaluated only with respect to the key space analysis for brute-force attacks and other limited attacks. Accordingly, in this paper, we evaluated the security of the method with respect to robustness against ciphertext-only attacks including state-of-the-art attacks. In experiments, the method is compared with conventional encryption methods, and it is confirmed to be robust against ciphertext-only attacks if parameters used for image encryption are carefully chosen.
△ Less
Submitted 8 October, 2024;
originally announced October 2024.
-
Enhancing Security Using Random Binary Weights in Privacy-Preserving Federated Learning
Authors:
Hiroto Sawada,
Shoko Imaizumi,
Hitoshi Kiya
Abstract:
In this paper, we propose a novel method for enhancing security in privacy-preserving federated learning using the Vision Transformer. In federated learning, learning is performed by collecting updated information without collecting raw data from each client. However, the problem is that this raw data may be inferred from the updated information. Conventional data-guessing countermeasures (securit…
▽ More
In this paper, we propose a novel method for enhancing security in privacy-preserving federated learning using the Vision Transformer. In federated learning, learning is performed by collecting updated information without collecting raw data from each client. However, the problem is that this raw data may be inferred from the updated information. Conventional data-guessing countermeasures (security enhancement methods) for addressing this issue have a trade-off relationship between privacy protection strength and learning efficiency, and they generally degrade model performance. In this paper, we propose a novel method of federated learning that does not degrade model performance and that is robust against data-guessing attacks on updated information. In the proposed method, each client independently prepares a sequence of binary (0 or 1) random numbers, multiplies it by the updated information, and sends it to a server for model learning. In experiments, the effectiveness of the proposed method is confirmed in terms of model performance and resistance to the APRIL (Attention PRIvacy Leakage) restoration attack.
△ Less
Submitted 30 September, 2024;
originally announced September 2024.
-
Privacy-Preserving Vision Transformer Using Images Encrypted with Restricted Random Permutation Matrices
Authors:
Kouki Horio,
Kiyoshi Nishikawa,
Hitoshi Kiya
Abstract:
We propose a novel method for privacy-preserving fine-tuning vision transformers (ViTs) with encrypted images. Conventional methods using encrypted images degrade model performance compared with that of using plain images due to the influence of image encryption. In contrast, the proposed encryption method using restricted random permutation matrices can provide a higher performance than the conve…
▽ More
We propose a novel method for privacy-preserving fine-tuning vision transformers (ViTs) with encrypted images. Conventional methods using encrypted images degrade model performance compared with that of using plain images due to the influence of image encryption. In contrast, the proposed encryption method using restricted random permutation matrices can provide a higher performance than the conventional ones.
△ Less
Submitted 16 August, 2024;
originally announced August 2024.
-
Disposable-key-based image encryption for collaborative learning of Vision Transformer
Authors:
Rei Aso,
Sayaka Shiota,
Hitoshi Kiya
Abstract:
We propose a novel method for securely training the vision transformer (ViT) with sensitive data shared from multiple clients similar to privacy-preserving federated learning. In the proposed method, training images are independently encrypted by each client where encryption keys can be prepared by each client, and ViT is trained by using these encrypted images for the first time. The method allow…
▽ More
We propose a novel method for securely training the vision transformer (ViT) with sensitive data shared from multiple clients similar to privacy-preserving federated learning. In the proposed method, training images are independently encrypted by each client where encryption keys can be prepared by each client, and ViT is trained by using these encrypted images for the first time. The method allows clients not only to dispose of the keys but to also reduce the communication costs between a central server and the clients. In image classification experiments, we verify the effectiveness of the proposed method on the CIFAR-10 dataset in terms of classification accuracy and the use of restricted random permutation matrices.
△ Less
Submitted 11 August, 2024;
originally announced August 2024.
-
Speech privacy-preserving methods using secret key for convolutional neural network models and their robustness evaluation
Authors:
Shoko Niwa,
Sayaka Shiota,
Hitoshi Kiya
Abstract:
In this paper, we propose privacy-preserving methods with a secret key for convolutional neural network (CNN)-based models in speech processing tasks. In environments where untrusted third parties, like cloud servers, provide CNN-based systems, ensuring the privacy of speech queries becomes essential. This paper proposes encryption methods for speech queries using secret keys and a model structure…
▽ More
In this paper, we propose privacy-preserving methods with a secret key for convolutional neural network (CNN)-based models in speech processing tasks. In environments where untrusted third parties, like cloud servers, provide CNN-based systems, ensuring the privacy of speech queries becomes essential. This paper proposes encryption methods for speech queries using secret keys and a model structure that allows for encrypted queries to be accepted without decryption. Our approach introduces three types of secret keys: Shuffling, Flipping, and random orthogonal matrix (ROM). In experiments, we demonstrate that when the proposed methods are used with the correct key, identification performance did not degrade. Conversely, when an incorrect key is used, the performance significantly decreased. Particularly, with the use of ROM, we show that even with a relatively small key space, high privacy-preserving performance can be maintained many speech processing tasks. Furthermore, we also demonstrate the difficulty of recovering original speech from encrypted queries in various robustness evaluations.
△ Less
Submitted 7 August, 2024;
originally announced August 2024.
-
Fine-Tuning Text-To-Image Diffusion Models for Class-Wise Spurious Feature Generation
Authors:
AprilPyone MaungMaung,
Huy H. Nguyen,
Hitoshi Kiya,
Isao Echizen
Abstract:
We propose a method for generating spurious features by leveraging large-scale text-to-image diffusion models. Although the previous work detects spurious features in a large-scale dataset like ImageNet and introduces Spurious ImageNet, we found that not all spurious images are spurious across different classifiers. Although spurious images help measure the reliance of a classifier, filtering many…
▽ More
We propose a method for generating spurious features by leveraging large-scale text-to-image diffusion models. Although the previous work detects spurious features in a large-scale dataset like ImageNet and introduces Spurious ImageNet, we found that not all spurious images are spurious across different classifiers. Although spurious images help measure the reliance of a classifier, filtering many images from the Internet to find more spurious features is time-consuming. To this end, we utilize an existing approach of personalizing large-scale text-to-image diffusion models with available discovered spurious images and propose a new spurious feature similarity loss based on neural features of an adversarially robust model. Precisely, we fine-tune Stable Diffusion with several reference images from Spurious ImageNet with a modified objective incorporating the proposed spurious-feature similarity loss. Experiment results show that our method can generate spurious images that are consistently spurious across different classifiers. Moreover, the generated spurious images are visually similar to reference images from Spurious ImageNet.
△ Less
Submitted 12 February, 2024;
originally announced February 2024.
-
A Random Ensemble of Encrypted Vision Transformers for Adversarially Robust Defense
Authors:
Ryota Iijima,
Sayaka Shiota,
Hitoshi Kiya
Abstract:
Deep neural networks (DNNs) are well known to be vulnerable to adversarial examples (AEs). In previous studies, the use of models encrypted with a secret key was demonstrated to be robust against white-box attacks, but not against black-box ones. In this paper, we propose a novel method using the vision transformer (ViT) that is a random ensemble of encrypted models for enhancing robustness agains…
▽ More
Deep neural networks (DNNs) are well known to be vulnerable to adversarial examples (AEs). In previous studies, the use of models encrypted with a secret key was demonstrated to be robust against white-box attacks, but not against black-box ones. In this paper, we propose a novel method using the vision transformer (ViT) that is a random ensemble of encrypted models for enhancing robustness against both white-box and black-box attacks. In addition, a benchmark attack method, called AutoAttack, is applied to models to test adversarial robustness objectively. In experiments, the method was demonstrated to be robust against not only white-box attacks but also black-box ones in an image classification task on the CIFAR-10 and ImageNet datasets. The method was also compared with the state-of-the-art in a standardized benchmark for adversarial robustness, RobustBench, and it was verified to outperform conventional defenses in terms of clean accuracy and robust accuracy.
△ Less
Submitted 11 February, 2024;
originally announced February 2024.
-
Efficient Fine-Tuning with Domain Adaptation for Privacy-Preserving Vision Transformer
Authors:
Teru Nagamori,
Sayaka Shiota,
Hitoshi Kiya
Abstract:
We propose a novel method for privacy-preserving deep neural networks (DNNs) with the Vision Transformer (ViT). The method allows us not only to train models and test with visually protected images but to also avoid the performance degradation caused from the use of encrypted images, whereas conventional methods cannot avoid the influence of image encryption. A domain adaptation method is used to…
▽ More
We propose a novel method for privacy-preserving deep neural networks (DNNs) with the Vision Transformer (ViT). The method allows us not only to train models and test with visually protected images but to also avoid the performance degradation caused from the use of encrypted images, whereas conventional methods cannot avoid the influence of image encryption. A domain adaptation method is used to efficiently fine-tune ViT with encrypted images. In experiments, the method is demonstrated to outperform conventional methods in an image classification task on the CIFAR-10 and ImageNet datasets in terms of classification accuracy.
△ Less
Submitted 9 February, 2024; v1 submitted 10 January, 2024;
originally announced January 2024.
-
A Random Ensemble of Encrypted models for Enhancing Robustness against Adversarial Examples
Authors:
Ryota Iijima,
Sayaka Shiota,
Hitoshi Kiya
Abstract:
Deep neural networks (DNNs) are well known to be vulnerable to adversarial examples (AEs). In addition, AEs have adversarial transferability, which means AEs generated for a source model can fool another black-box model (target model) with a non-trivial probability. In previous studies, it was confirmed that the vision transformer (ViT) is more robust against the property of adversarial transferab…
▽ More
Deep neural networks (DNNs) are well known to be vulnerable to adversarial examples (AEs). In addition, AEs have adversarial transferability, which means AEs generated for a source model can fool another black-box model (target model) with a non-trivial probability. In previous studies, it was confirmed that the vision transformer (ViT) is more robust against the property of adversarial transferability than convolutional neural network (CNN) models such as ConvMixer, and moreover encrypted ViT is more robust than ViT without any encryption. In this article, we propose a random ensemble of encrypted ViT models to achieve much more robust models. In experiments, the proposed scheme is verified to be more robust against not only black-box attacks but also white-box ones than convention methods.
△ Less
Submitted 4 January, 2024;
originally announced January 2024.
-
Efficient Key-Based Adversarial Defense for ImageNet by Using Pre-trained Model
Authors:
AprilPyone MaungMaung,
Isao Echizen,
Hitoshi Kiya
Abstract:
In this paper, we propose key-based defense model proliferation by leveraging pre-trained models and utilizing recent efficient fine-tuning techniques on ImageNet-1k classification. First, we stress that deploying key-based models on edge devices is feasible with the latest model deployment advancements, such as Apple CoreML, although the mainstream enterprise edge artificial intelligence (Edge AI…
▽ More
In this paper, we propose key-based defense model proliferation by leveraging pre-trained models and utilizing recent efficient fine-tuning techniques on ImageNet-1k classification. First, we stress that deploying key-based models on edge devices is feasible with the latest model deployment advancements, such as Apple CoreML, although the mainstream enterprise edge artificial intelligence (Edge AI) has been focused on the Cloud. Then, we point out that the previous key-based defense on on-device image classification is impractical for two reasons: (1) training many classifiers from scratch is not feasible, and (2) key-based defenses still need to be thoroughly tested on large datasets like ImageNet. To this end, we propose to leverage pre-trained models and utilize efficient fine-tuning techniques to proliferate key-based models even on limited computing resources. Experiments were carried out on the ImageNet-1k dataset using adaptive and non-adaptive attacks. The results show that our proposed fine-tuned key-based models achieve a superior classification accuracy (more than 10% increase) compared to the previous key-based models on classifying clean and adversarial examples.
△ Less
Submitted 28 November, 2023;
originally announced November 2023.
-
Turning Tiles is PSPACE-complete
Authors:
Kanae Yoshiwatari,
Hironori Kiya,
Koki Suetsugu,
Tesshu Hanaka,
Hirotaka Ono
Abstract:
In combinatorial game theory, the winning player for a position in normal play is analyzed and characterized via algebraic operations. Such analyses define a value for each position, called a game value. A game (ruleset) is called universal if any game value is achievable in some position in a play of the game. Although the universality of a game implies that the ruleset is rich enough (i.e., suff…
▽ More
In combinatorial game theory, the winning player for a position in normal play is analyzed and characterized via algebraic operations. Such analyses define a value for each position, called a game value. A game (ruleset) is called universal if any game value is achievable in some position in a play of the game. Although the universality of a game implies that the ruleset is rich enough (i.e., sufficiently complex), it does not immediately imply that the game is intractable in the sense of computational complexity. This paper proves that the universal game Turning Tiles is PSPACE-complete.
△ Less
Submitted 3 October, 2023;
originally announced October 2023.
-
Domain Adaptation for Efficiently Fine-tuning Vision Transformer with Encrypted Images
Authors:
Teru Nagamori,
Sayaka Shiota,
Hitoshi Kiya
Abstract:
In recent years, deep neural networks (DNNs) trained with transformed data have been applied to various applications such as privacy-preserving learning, access control, and adversarial defenses. However, the use of transformed data decreases the performance of models. Accordingly, in this paper, we propose a novel method for fine-tuning models with transformed images under the use of the vision t…
▽ More
In recent years, deep neural networks (DNNs) trained with transformed data have been applied to various applications such as privacy-preserving learning, access control, and adversarial defenses. However, the use of transformed data decreases the performance of models. Accordingly, in this paper, we propose a novel method for fine-tuning models with transformed images under the use of the vision transformer (ViT). The proposed domain adaptation method does not cause the accuracy degradation of models, and it is carried out on the basis of the embedding structure of ViT. In experiments, we confirmed that the proposed method prevents accuracy degradation even when using encrypted images with the CIFAR-10 and CIFAR-100 datasets.
△ Less
Submitted 6 September, 2023; v1 submitted 5 September, 2023;
originally announced September 2023.
-
Hindering Adversarial Attacks with Multiple Encrypted Patch Embeddings
Authors:
AprilPyone MaungMaung,
Isao Echizen,
Hitoshi Kiya
Abstract:
In this paper, we propose a new key-based defense focusing on both efficiency and robustness. Although the previous key-based defense seems effective in defending against adversarial examples, carefully designed adaptive attacks can bypass the previous defense, and it is difficult to train the previous defense on large datasets like ImageNet. We build upon the previous defense with two major impro…
▽ More
In this paper, we propose a new key-based defense focusing on both efficiency and robustness. Although the previous key-based defense seems effective in defending against adversarial examples, carefully designed adaptive attacks can bypass the previous defense, and it is difficult to train the previous defense on large datasets like ImageNet. We build upon the previous defense with two major improvements: (1) efficient training and (2) optional randomization. The proposed defense utilizes one or more secret patch embeddings and classifier heads with a pre-trained isotropic network. When more than one secret embeddings are used, the proposed defense enables randomization on inference. Experiments were carried out on the ImageNet dataset, and the proposed defense was evaluated against an arsenal of state-of-the-art attacks, including adaptive ones. The results show that the proposed defense achieves a high robust accuracy and a comparable clean accuracy compared to the previous key-based defense.
△ Less
Submitted 4 September, 2023;
originally announced September 2023.
-
Block-Wise Encryption for Reliable Vision Transformer models
Authors:
Hitoshi Kiya,
Ryota Iijima,
Teru Nagamori
Abstract:
This article presents block-wise image encryption for the vision transformer and its applications. Perceptual image encryption for deep learning enables us not only to protect the visual information of plain images but to also embed unique features controlled with a key into images and models. However, when using conventional perceptual encryption methods, the performance of models is degraded due…
▽ More
This article presents block-wise image encryption for the vision transformer and its applications. Perceptual image encryption for deep learning enables us not only to protect the visual information of plain images but to also embed unique features controlled with a key into images and models. However, when using conventional perceptual encryption methods, the performance of models is degraded due to the influence of encryption. In this paper, we focus on block-wise encryption for the vision transformer, and we introduce three applications: privacy-preserving image classification, access control, and the combined use of federated learning and encrypted images. Our scheme can have the same performance as models without any encryption, and it does not require any network modification. It also allows us to easily update the secret key. In experiments, the effectiveness of the scheme is demonstrated in terms of performance degradation and access control on the CIFAR10 and CIFAR-100 datasets.
△ Less
Submitted 15 August, 2023;
originally announced August 2023.
-
Security Evaluation of Compressible and Learnable Image Encryption Against Jigsaw Puzzle Solver Attacks
Authors:
Tatsuya Chuman,
Nobutaka Ono,
Hitoshi Kiya
Abstract:
Several learnable image encryption schemes have been developed for privacy-preserving image classification. This paper focuses on the security block-based image encryption methods that are learnable and JPEG-friendly. Permuting divided blocks in an image is known to enhance robustness against ciphertext-only attacks (COAs), but recently jigsaw puzzle solver attacks have been demonstrated to be abl…
▽ More
Several learnable image encryption schemes have been developed for privacy-preserving image classification. This paper focuses on the security block-based image encryption methods that are learnable and JPEG-friendly. Permuting divided blocks in an image is known to enhance robustness against ciphertext-only attacks (COAs), but recently jigsaw puzzle solver attacks have been demonstrated to be able to restore visual information on the encrypted images. In contrast, it has never been confirmed whether encrypted images including noise caused by JPEG-compression are robust. Accordingly, the aim of this paper is to evaluate the security of compressible and learnable encrypted images against jigsaw puzzle solver attacks. In experiments, the security evaluation was carried out on the CIFAR-10 and STL-10 datasets under JPEG-compression.
△ Less
Submitted 4 August, 2023;
originally announced August 2023.
-
Enhanced Security with Encrypted Vision Transformer in Federated Learning
Authors:
Rei Aso,
Sayaka Shiota,
Hitoshi Kiya
Abstract:
Federated learning is a learning method for training models over multiple participants without directly sharing their raw data, and it has been expected to be a privacy protection method for training data. In contrast, attack methods have been studied to restore learning data from model information shared with clients, so enhanced security against attacks has become an urgent problem. Accordingly,…
▽ More
Federated learning is a learning method for training models over multiple participants without directly sharing their raw data, and it has been expected to be a privacy protection method for training data. In contrast, attack methods have been studied to restore learning data from model information shared with clients, so enhanced security against attacks has become an urgent problem. Accordingly, in this article, we propose a novel framework of federated learning on the bases of the embedded structure of the vision transformer by using the model information encrypted with a random sequence. In image classification experiments, we verify the effectiveness of the proposed method on the CIFAR-10 dataset in terms of classification accuracy and robustness against attacks.
△ Less
Submitted 1 August, 2023;
originally announced August 2023.
-
Enhanced Security against Adversarial Examples Using a Random Ensemble of Encrypted Vision Transformer Models
Authors:
Ryota Iijima,
Miki Tanaka,
Sayaka Shiota,
Hitoshi Kiya
Abstract:
Deep neural networks (DNNs) are well known to be vulnerable to adversarial examples (AEs). In addition, AEs have adversarial transferability, which means AEs generated for a source model can fool another black-box model (target model) with a non-trivial probability. In previous studies, it was confirmed that the vision transformer (ViT) is more robust against the property of adversarial transferab…
▽ More
Deep neural networks (DNNs) are well known to be vulnerable to adversarial examples (AEs). In addition, AEs have adversarial transferability, which means AEs generated for a source model can fool another black-box model (target model) with a non-trivial probability. In previous studies, it was confirmed that the vision transformer (ViT) is more robust against the property of adversarial transferability than convolutional neural network (CNN) models such as ConvMixer, and moreover encrypted ViT is more robust than ViT without any encryption. In this article, we propose a random ensemble of encrypted ViT models to achieve much more robust models. In experiments, the proposed scheme is verified to be more robust against not only black-box attacks but also white-box ones than convention methods.
△ Less
Submitted 26 July, 2023;
originally announced July 2023.
-
Generative Model-Based Attack on Learnable Image Encryption for Privacy-Preserving Deep Learning
Authors:
AprilPyone MaungMaung,
Hitoshi Kiya
Abstract:
In this paper, we propose a novel generative model-based attack on learnable image encryption methods proposed for privacy-preserving deep learning. Various learnable encryption methods have been studied to protect the sensitive visual information of plain images, and some of them have been investigated to be robust enough against all existing attacks. However, previous attacks on image encryption…
▽ More
In this paper, we propose a novel generative model-based attack on learnable image encryption methods proposed for privacy-preserving deep learning. Various learnable encryption methods have been studied to protect the sensitive visual information of plain images, and some of them have been investigated to be robust enough against all existing attacks. However, previous attacks on image encryption focus only on traditional cryptanalytic attacks or reverse translation models, so these attacks cannot recover any visual information if a block-scrambling encryption step, which effectively destroys global information, is applied. Accordingly, in this paper, generative models are explored to evaluate whether such models can restore sensitive visual information from encrypted images for the first time. We first point out that encrypted images have some similarity with plain images in the embedding space. By taking advantage of leaked information from encrypted images, we propose a guided generative model as an attack on learnable image encryption to recover personally identifiable visual information. We implement the proposed attack in two ways by utilizing two state-of-the-art generative models: a StyleGAN-based model and latent diffusion-based one. Experiments were carried out on the CelebA-HQ and ImageNet datasets. Results show that images reconstructed by the proposed method have perceptual similarities to plain images.
△ Less
Submitted 9 March, 2023;
originally announced March 2023.
-
Combined Use of Federated Learning and Image Encryption for Privacy-Preserving Image Classification with Vision Transformer
Authors:
Teru Nagamori,
Hitoshi Kiya
Abstract:
In recent years, privacy-preserving methods for deep learning have become an urgent problem. Accordingly, we propose the combined use of federated learning (FL) and encrypted images for privacy-preserving image classification under the use of the vision transformer (ViT). The proposed method allows us not only to train models over multiple participants without directly sharing their raw data but t…
▽ More
In recent years, privacy-preserving methods for deep learning have become an urgent problem. Accordingly, we propose the combined use of federated learning (FL) and encrypted images for privacy-preserving image classification under the use of the vision transformer (ViT). The proposed method allows us not only to train models over multiple participants without directly sharing their raw data but to also protect the privacy of test (query) images for the first time. In addition, it can also maintain the same accuracy as normally trained models. In an experiment, the proposed method was demonstrated to well work without any performance degradation on the CIFAR-10 and CIFAR-100 datasets.
△ Less
Submitted 3 March, 2023; v1 submitted 22 January, 2023;
originally announced January 2023.
-
Color-NeuraCrypt: Privacy-Preserving Color-Image Classification Using Extended Random Neural Networks
Authors:
Zheng Qi,
AprilPyone MaungMaung,
Hitoshi Kiya
Abstract:
In recent years, with the development of cloud computing platforms, privacy-preserving methods for deep learning have become an urgent problem. NeuraCrypt is a private random neural network for privacy-preserving that allows data owners to encrypt the medical data before the data uploading, and data owners can train and then test their models in a cloud server with the encrypted data directly. How…
▽ More
In recent years, with the development of cloud computing platforms, privacy-preserving methods for deep learning have become an urgent problem. NeuraCrypt is a private random neural network for privacy-preserving that allows data owners to encrypt the medical data before the data uploading, and data owners can train and then test their models in a cloud server with the encrypted data directly. However, we point out that the performance of NeuraCrypt is heavily degraded when using color images. In this paper, we propose a Color-NeuraCrypt to solve this problem. Experiment results show that our proposed Color-NeuraCrypt can achieve a better classification accuracy than the original one and other privacy-preserving methods.
△ Less
Submitted 12 January, 2023;
originally announced January 2023.
-
A Privacy Preserving Method with a Random Orthogonal Matrix for ConvMixer Models
Authors:
Rei Aso,
Tatsuya Chuman,
Hitoshi Kiya
Abstract:
In this paper, a privacy preserving image classification method is proposed under the use of ConvMixer models. To protect the visual information of test images, a test image is divided into blocks, and then every block is encrypted by using a random orthogonal matrix. Moreover, a ConvMixer model trained with plain images is transformed by the random orthogonal matrix used for encrypting test image…
▽ More
In this paper, a privacy preserving image classification method is proposed under the use of ConvMixer models. To protect the visual information of test images, a test image is divided into blocks, and then every block is encrypted by using a random orthogonal matrix. Moreover, a ConvMixer model trained with plain images is transformed by the random orthogonal matrix used for encrypting test images, on the basis of the embedding structure of ConvMixer. The proposed method allows us not only to use the same classification accuracy as that of ConvMixer models without considering privacy protection but to also enhance robustness against various attacks compared to conventional privacy-preserving learning.
△ Less
Submitted 17 January, 2023; v1 submitted 10 January, 2023;
originally announced January 2023.
-
Winner Determination Algorithms for Graph Games with Matching Structures
Authors:
Tesshu Hanaka,
Hironori Kiya,
Hirotaka Ono,
Kanae Yoshiwatari
Abstract:
Cram, Domineering, and Arc Kayles are well-studied combinatorial games. They are interpreted as edge-selecting-type games on graphs, and the selected edges during a game form a matching. In this paper, we define a generalized game called Colored Arc Kayles, which includes these games. Colored Arc Kayles is played on a graph whose edges are colored in black, white, or gray, and black (resp., white)…
▽ More
Cram, Domineering, and Arc Kayles are well-studied combinatorial games. They are interpreted as edge-selecting-type games on graphs, and the selected edges during a game form a matching. In this paper, we define a generalized game called Colored Arc Kayles, which includes these games. Colored Arc Kayles is played on a graph whose edges are colored in black, white, or gray, and black (resp., white) edges can be selected only by the black (resp., white) player, although gray edges can be selected by both black and white players. We first observe that the winner determination for Colored Arc Kayles can be done in $O^*(2^n)$ time by a simple algorithm, where $n$ is the order of a graph. We then focus on the vertex cover number, which is linearly related to the number of turns, and show that Colored Arc Kayles, BW-Arc Kayles, and Arc Kayles are solved in time $O^*(1.4143^{τ^2+3.17τ})$, $O^*(1.3161^{τ^2+4τ})$, and $O^*(1.1893^{τ^2+6.34τ})$, respectively, where $τ$ is the vertex cover number. Furthermore, we present an $O^*((n/ν+1)^ν)$-time algorithm for Arc Kayles, where $ν$ is neighborhood diversity. We finally show that Arc Kayles on trees can be solved in $O^* (2^{n/2})(=O(1.4143^n))$ time, which improves $O^*(3^{n/3})(=O(1.4423^n))$ by a direct adjustment of the analysis of Bodlaender et al.'s $O^*(3^{n/3})$-time algorithm for Node Kayles.
△ Less
Submitted 9 November, 2022;
originally announced November 2022.
-
A Jigsaw Puzzle Solver-based Attack on Block-wise Image Encryption for Privacy-preserving DNNs
Authors:
Tatsuya Chuman,
Hitoshi Kiya
Abstract:
Privacy-preserving deep neural networks (DNNs) have been proposed for protecting data privacy in the cloud server. Although several encryption schemes for visually protection have been proposed for privacy-preserving DNNs, several attacks enable to restore visual information from encrypted images. On the other hand, it has been confirmed that the block-wise image encryption scheme which utilizes b…
▽ More
Privacy-preserving deep neural networks (DNNs) have been proposed for protecting data privacy in the cloud server. Although several encryption schemes for visually protection have been proposed for privacy-preserving DNNs, several attacks enable to restore visual information from encrypted images. On the other hand, it has been confirmed that the block-wise image encryption scheme which utilizes block and pixel shuffling is robust against several attacks. In this paper, we propose a jigsaw puzzle solver-based attack to restore visual information from encrypted images including block and pixel shuffling. In experiments, images encrypted by using the block-wise image encryption are mostly restored by using the proposed attack.
△ Less
Submitted 10 November, 2022; v1 submitted 4 November, 2022;
originally announced November 2022.
-
Sequentially Swapping Tokens: Further on Graph Classes
Authors:
Hironori Kiya,
Yuto Okada,
Hirotaka Ono,
Yota Otachi
Abstract:
We study the following variant of the 15 puzzle. Given a graph and two token placements on the vertices, we want to find a walk of the minimum length (if any exists) such that the sequence of token swappings along the walk obtains one of the given token placements from the other one. This problem was introduced as Sequential Token Swapping by Yamanaka et al. [JGAA 2019], who showed that the proble…
▽ More
We study the following variant of the 15 puzzle. Given a graph and two token placements on the vertices, we want to find a walk of the minimum length (if any exists) such that the sequence of token swappings along the walk obtains one of the given token placements from the other one. This problem was introduced as Sequential Token Swapping by Yamanaka et al. [JGAA 2019], who showed that the problem is intractable in general but polynomial-time solvable for trees, complete graphs, and cycles. In this paper, we present a polynomial-time algorithm for block-cactus graphs, which include all previously known cases. We also present general tools for showing the hardness of the problem on restricted graph classes such as chordal graphs and chordal bipartite graphs. We also show that the problem is hard on grids and king's graphs, which are the graphs corresponding to the 15 puzzle and its variant with relaxed moves.
△ Less
Submitted 9 March, 2023; v1 submitted 6 October, 2022;
originally announced October 2022.
-
Access Control with Encrypted Feature Maps for Object Detection Models
Authors:
Teru Nagamori,
Hiroki Ito,
AprilPyone MaungMaung,
Hitoshi Kiya
Abstract:
In this paper, we propose an access control method with a secret key for object detection models for the first time so that unauthorized users without a secret key cannot benefit from the performance of trained models. The method enables us not only to provide a high detection performance to authorized users but to also degrade the performance for unauthorized users. The use of transformed images…
▽ More
In this paper, we propose an access control method with a secret key for object detection models for the first time so that unauthorized users without a secret key cannot benefit from the performance of trained models. The method enables us not only to provide a high detection performance to authorized users but to also degrade the performance for unauthorized users. The use of transformed images was proposed for the access control of image classification models, but these images cannot be used for object detection models due to performance degradation. Accordingly, in this paper, selected feature maps are encrypted with a secret key for training and testing models, instead of input images. In an experiment, the protected models allowed authorized users to obtain almost the same performance as that of non-protected models but also with robustness against unauthorized access without a key.
△ Less
Submitted 29 September, 2022;
originally announced September 2022.
-
On the Adversarial Transferability of ConvMixer Models
Authors:
Ryota Iijima,
Miki Tanaka,
Isao Echizen,
Hitoshi Kiya
Abstract:
Deep neural networks (DNNs) are well known to be vulnerable to adversarial examples (AEs). In addition, AEs have adversarial transferability, which means AEs generated for a source model can fool another black-box model (target model) with a non-trivial probability. In this paper, we investigate the property of adversarial transferability between models including ConvMixer, which is an isotropic n…
▽ More
Deep neural networks (DNNs) are well known to be vulnerable to adversarial examples (AEs). In addition, AEs have adversarial transferability, which means AEs generated for a source model can fool another black-box model (target model) with a non-trivial probability. In this paper, we investigate the property of adversarial transferability between models including ConvMixer, which is an isotropic network, for the first time. To objectively verify the property of transferability, the robustness of models is evaluated by using a benchmark attack method called AutoAttack. In an image classification experiment, ConvMixer is confirmed to be weak to adversarial transferability.
△ Less
Submitted 18 September, 2022;
originally announced September 2022.
-
StyleGAN Encoder-Based Attack for Block Scrambled Face Images
Authors:
AprilPyone MaungMaung,
Hitoshi Kiya
Abstract:
In this paper, we propose an attack method to block scrambled face images, particularly Encryption-then-Compression (EtC) applied images by utilizing the existing powerful StyleGAN encoder and decoder for the first time. Instead of reconstructing identical images as plain ones from encrypted images, we focus on recovering styles that can reveal identifiable information from the encrypted images. T…
▽ More
In this paper, we propose an attack method to block scrambled face images, particularly Encryption-then-Compression (EtC) applied images by utilizing the existing powerful StyleGAN encoder and decoder for the first time. Instead of reconstructing identical images as plain ones from encrypted images, we focus on recovering styles that can reveal identifiable information from the encrypted images. The proposed method trains an encoder by using plain and encrypted image pairs with a particular training strategy. While state-of-the-art attack methods cannot recover any perceptual information from EtC images, the proposed method discloses personally identifiable information such as hair color, skin color, eyeglasses, gender, etc. Experiments were carried out on the CelebA dataset, and results show that reconstructed images have some perceptual similarities compared to plain images.
△ Less
Submitted 16 September, 2022;
originally announced September 2022.
-
On the Transferability of Adversarial Examples between Encrypted Models
Authors:
Miki Tanaka,
Isao Echizen,
Hitoshi Kiya
Abstract:
Deep neural networks (DNNs) are well known to be vulnerable to adversarial examples (AEs). In addition, AEs have adversarial transferability, namely, AEs generated for a source model fool other (target) models. In this paper, we investigate the transferability of models encrypted for adversarially robust defense for the first time. To objectively verify the property of transferability, the robustn…
▽ More
Deep neural networks (DNNs) are well known to be vulnerable to adversarial examples (AEs). In addition, AEs have adversarial transferability, namely, AEs generated for a source model fool other (target) models. In this paper, we investigate the transferability of models encrypted for adversarially robust defense for the first time. To objectively verify the property of transferability, the robustness of models is evaluated by using a benchmark attack method, called AutoAttack. In an image-classification experiment, the use of encrypted models is confirmed not only to be robust against AEs but to also reduce the influence of AEs in terms of the transferability of models.
△ Less
Submitted 7 September, 2022;
originally announced September 2022.
-
An Access Control Method with Secret Key for Semantic Segmentation Models
Authors:
Teru Nagamori,
Ryota Iijima,
Hitoshi Kiya
Abstract:
A novel method for access control with a secret key is proposed to protect models from unauthorized access in this paper. We focus on semantic segmentation models with the vision transformer (ViT), called segmentation transformer (SETR). Most existing access control methods focus on image classification tasks, or they are limited to CNNs. By using a patch embedding structure that ViT has, trained…
▽ More
A novel method for access control with a secret key is proposed to protect models from unauthorized access in this paper. We focus on semantic segmentation models with the vision transformer (ViT), called segmentation transformer (SETR). Most existing access control methods focus on image classification tasks, or they are limited to CNNs. By using a patch embedding structure that ViT has, trained models and test images can be efficiently encrypted with a secret key, and then semantic segmentation tasks are carried out in the encrypted domain. In an experiment, the method is confirmed to provide the same accuracy as that of using plain images without any encryption to authorized users with a correct key and also to provide an extremely degraded accuracy to unauthorized users.
△ Less
Submitted 28 August, 2022;
originally announced August 2022.
-
A Detection Method of Temporally Operated Videos Using Robust Hashing
Authors:
Shoko Niwa,
Miki Tanaka,
Hitoshi Kiya
Abstract:
SNS providers are known to carry out the recompression and resizing of uploaded videos/images, but most conventional methods for detecting tampered videos/images are not robust enough against such operations. In addition, videos are temporally operated such as the insertion of new frames and the permutation of frames, of which operations are difficult to be detected by using conventional methods.…
▽ More
SNS providers are known to carry out the recompression and resizing of uploaded videos/images, but most conventional methods for detecting tampered videos/images are not robust enough against such operations. In addition, videos are temporally operated such as the insertion of new frames and the permutation of frames, of which operations are difficult to be detected by using conventional methods. Accordingly, in this paper, we propose a novel method with a robust hashing algorithm for detecting temporally operated videos even when applying resizing and compression to the videos.
△ Less
Submitted 11 August, 2022; v1 submitted 10 August, 2022;
originally announced August 2022.
-
Privacy-Preserving Image Classification Using ConvMixer with Adaptive Permutation Matrix
Authors:
Zheng Qi,
AprilPyone MaungMaung,
Hitoshi Kiya
Abstract:
In this paper, we propose a privacy-preserving image classification method using encrypted images under the use of the ConvMixer structure. Block-wise scrambled images, which are robust enough against various attacks, have been used for privacy-preserving image classification tasks, but the combined use of a classification network and an adaptation network is needed to reduce the influence of imag…
▽ More
In this paper, we propose a privacy-preserving image classification method using encrypted images under the use of the ConvMixer structure. Block-wise scrambled images, which are robust enough against various attacks, have been used for privacy-preserving image classification tasks, but the combined use of a classification network and an adaptation network is needed to reduce the influence of image encryption. However, images with a large size cannot be applied to the conventional method with an adaptation network because the adaptation network has so many parameters. Accordingly, we propose a novel method, which allows us not only to apply block-wise scrambled images to ConvMixer for both training and testing without the adaptation network, but also to provide a higher classification accuracy than conventional methods.
△ Less
Submitted 4 August, 2022;
originally announced August 2022.
-
Template matching with white balance adjustment under multiple illuminants
Authors:
Teruaki Akazawa,
Yuma Kinoshita,
Hitoshi Kiya
Abstract:
In this paper, we propose a novel template matching method with a white balancing adjustment, called N-white balancing, which was proposed for multi-illuminant scenes. To reduce the influence of lighting effects, N-white balancing is applied to images for multi-illumination color constancy, and then a template matching method is carried out by using adjusted images. In experiments, the effectivene…
▽ More
In this paper, we propose a novel template matching method with a white balancing adjustment, called N-white balancing, which was proposed for multi-illuminant scenes. To reduce the influence of lighting effects, N-white balancing is applied to images for multi-illumination color constancy, and then a template matching method is carried out by using adjusted images. In experiments, the effectiveness of the proposed method is demonstrated to be effective in object detection tasks under various illumination conditions.
△ Less
Submitted 3 August, 2022;
originally announced August 2022.
-
An Encryption Method of ConvMixer Models without Performance Degradation
Authors:
Ryota Iijima,
Hitoshi Kiya
Abstract:
In this paper, we propose an encryption method for ConvMixer models with a secret key. Encryption methods for DNN models have been studied to achieve adversarial defense, model protection and privacy-preserving image classification. However, the use of conventional encryption methods degrades the performance of models compared with that of plain models. Accordingly, we propose a novel method for e…
▽ More
In this paper, we propose an encryption method for ConvMixer models with a secret key. Encryption methods for DNN models have been studied to achieve adversarial defense, model protection and privacy-preserving image classification. However, the use of conventional encryption methods degrades the performance of models compared with that of plain models. Accordingly, we propose a novel method for encrypting ConvMixer models. The method is carried out on the basis of an embedding architecture that ConvMixer has, and models encrypted with the method can have the same performance as models trained with plain images only when using test images encrypted with a secret key. In addition, the proposed method does not require any specially prepared data for model training or network modification. In an experiment, the effectiveness of the proposed method is evaluated in terms of classification accuracy and model protection in an image classification task on the CIFAR10 dataset.
△ Less
Submitted 25 July, 2022;
originally announced July 2022.
-
Security Evaluation of Compressible Image Encryption for Privacy-Preserving Image Classification against Ciphertext-only Attacks
Authors:
Tatsuya Chuman,
Hitoshi Kiya
Abstract:
The security of learnable image encryption schemes for image classification using deep neural networks against several attacks has been discussed. On the other hand, block scrambling image encryption using the vision transformer has been proposed, which applies to lossless compression methods such as JPEG standard by dividing an image into permuted blocks. Although robustness of the block scrambli…
▽ More
The security of learnable image encryption schemes for image classification using deep neural networks against several attacks has been discussed. On the other hand, block scrambling image encryption using the vision transformer has been proposed, which applies to lossless compression methods such as JPEG standard by dividing an image into permuted blocks. Although robustness of the block scrambling image encryption against jigsaw puzzle solver attacks that utilize a correlation among the blocks has been evaluated under the condition of a large number of encrypted blocks, the security of encrypted images with a small number of blocks has never been evaluated. In this paper, the security of the block scrambling image encryption against ciphertext-only attacks is evaluated by using jigsaw puzzle solver attacks.
△ Less
Submitted 17 July, 2022;
originally announced July 2022.
-
Image and Model Transformation with Secret Key for Vision Transformer
Authors:
Hitoshi Kiya,
Ryota Iijima,
MaungMaung Aprilpyone,
Yuma Kinoshita
Abstract:
In this paper, we propose a combined use of transformed images and vision transformer (ViT) models transformed with a secret key. We show for the first time that models trained with plain images can be directly transformed to models trained with encrypted images on the basis of the ViT architecture, and the performance of the transformed models is the same as models trained with plain images when…
▽ More
In this paper, we propose a combined use of transformed images and vision transformer (ViT) models transformed with a secret key. We show for the first time that models trained with plain images can be directly transformed to models trained with encrypted images on the basis of the ViT architecture, and the performance of the transformed models is the same as models trained with plain images when using test images encrypted with the key. In addition, the proposed scheme does not require any specially prepared data for training models or network modification, so it also allows us to easily update the secret key. In an experiment, the effectiveness of the proposed scheme is evaluated in terms of performance degradation and model protection performance in an image classification task on the CIFAR-10 dataset.
△ Less
Submitted 20 July, 2022; v1 submitted 12 July, 2022;
originally announced July 2022.
-
Access Control of Semantic Segmentation Models Using Encrypted Feature Maps
Authors:
Hiroki Ito,
AprilPyone MaungMaung,
Sayaka Shiota,
Hitoshi Kiya
Abstract:
In this paper, we propose an access control method with a secret key for semantic segmentation models for the first time so that unauthorized users without a secret key cannot benefit from the performance of trained models. The method enables us not only to provide a high segmentation performance to authorized users but to also degrade the performance for unauthorized users. We first point out tha…
▽ More
In this paper, we propose an access control method with a secret key for semantic segmentation models for the first time so that unauthorized users without a secret key cannot benefit from the performance of trained models. The method enables us not only to provide a high segmentation performance to authorized users but to also degrade the performance for unauthorized users. We first point out that, for the application of semantic segmentation, conventional access control methods which use encrypted images for classification tasks are not directly applicable due to performance degradation. Accordingly, in this paper, selected feature maps are encrypted with a secret key for training and testing models, instead of input images. In an experiment, the protected models allowed authorized users to obtain almost the same performance as that of non-protected models but also with robustness against unauthorized access without a key.
△ Less
Submitted 11 June, 2022;
originally announced June 2022.
-
Privacy-Preserving Image Classification Using Vision Transformer
Authors:
Zheng Qi,
AprilPyone MaungMaung,
Yuma Kinoshita,
Hitoshi Kiya
Abstract:
In this paper, we propose a privacy-preserving image classification method that is based on the combined use of encrypted images and the vision transformer (ViT). The proposed method allows us not only to apply images without visual information to ViT models for both training and testing but to also maintain a high classification accuracy. ViT utilizes patch embedding and position embedding for im…
▽ More
In this paper, we propose a privacy-preserving image classification method that is based on the combined use of encrypted images and the vision transformer (ViT). The proposed method allows us not only to apply images without visual information to ViT models for both training and testing but to also maintain a high classification accuracy. ViT utilizes patch embedding and position embedding for image patches, so this architecture is shown to reduce the influence of block-wise image transformation. In an experiment, the proposed method for privacy-preserving image classification is demonstrated to outperform state-of-the-art methods in terms of classification accuracy and robustness against various attacks.
△ Less
Submitted 24 May, 2022;
originally announced May 2022.
-
Privacy-Preserving Image Classification Using Isotropic Network
Authors:
AprilPyone MaungMaung,
Hitoshi Kiya
Abstract:
In this paper, we propose a privacy-preserving image classification method that uses encrypted images and an isotropic network such as the vision transformer. The proposed method allows us not only to apply images without visual information to deep neural networks (DNNs) for both training and testing but also to maintain a high classification accuracy. In addition, compressible encrypted images, c…
▽ More
In this paper, we propose a privacy-preserving image classification method that uses encrypted images and an isotropic network such as the vision transformer. The proposed method allows us not only to apply images without visual information to deep neural networks (DNNs) for both training and testing but also to maintain a high classification accuracy. In addition, compressible encrypted images, called encryption-then-compression (EtC) images, can be used for both training and testing without any adaptation network. Previously, to classify EtC images, an adaptation network was required before a classification network, so methods with an adaptation network have been only tested on small images. To the best of our knowledge, previous privacy-preserving image classification methods have never considered image compressibility and patch embedding-based isotropic networks. In an experiment, the proposed privacy-preserving image classification was demonstrated to outperform state-of-the-art methods even when EtC images were used in terms of classification accuracy and robustness against various attacks under the use of two isotropic networks: vision transformer and ConvMixer.
△ Less
Submitted 15 April, 2022;
originally announced April 2022.
-
On the predictability in reversible steganography
Authors:
Ching-Chun Chang,
Xu Wang,
Sisheng Chen,
Hitoshi Kiya,
Isao Echizen
Abstract:
Artificial neural networks have advanced the frontiers of reversible steganography. The core strength of neural networks is the ability to render accurate predictions for a bewildering variety of data. Residual modulation is recognised as the most advanced reversible steganographic algorithm for digital images. The pivot of this algorithm is predictive analytics in which pixel intensities are pred…
▽ More
Artificial neural networks have advanced the frontiers of reversible steganography. The core strength of neural networks is the ability to render accurate predictions for a bewildering variety of data. Residual modulation is recognised as the most advanced reversible steganographic algorithm for digital images. The pivot of this algorithm is predictive analytics in which pixel intensities are predicted given some pixel-wise contextual information. This task can be perceived as a low-level vision problem and hence neural networks for addressing a similar class of problems can be deployed. On top of the prior art, this paper investigates predictability of pixel intensities based on supervised and unsupervised learning frameworks. Predictability analysis enables adaptive data embedding, which in turn leads to a better trade-off between capacity and imperceptibility. While conventional methods estimate predictability by the statistics of local image patterns, learning-based frameworks consider further the degree to which correct predictions can be made by a designated predictor. Not only should the image patterns be taken into account but also the predictor in use. Experimental results show that steganographic performance can be significantly improved by incorporating the learning-based predictability analysers into a reversible steganographic system.
△ Less
Submitted 7 March, 2023; v1 submitted 5 February, 2022;
originally announced February 2022.
-
Adversarial Detector with Robust Classifier
Authors:
Takayuki Osakabe,
Maungmaung Aprilpyone,
Sayaka Shiota,
Hitoshi Kiya
Abstract:
Deep neural network (DNN) models are wellknown to easily misclassify prediction results by using input images with small perturbations, called adversarial examples. In this paper, we propose a novel adversarial detector, which consists of a robust classifier and a plain one, to highly detect adversarial examples. The proposed adversarial detector is carried out in accordance with the logits of pla…
▽ More
Deep neural network (DNN) models are wellknown to easily misclassify prediction results by using input images with small perturbations, called adversarial examples. In this paper, we propose a novel adversarial detector, which consists of a robust classifier and a plain one, to highly detect adversarial examples. The proposed adversarial detector is carried out in accordance with the logits of plain and robust classifiers. In an experiment, the proposed detector is demonstrated to outperform a state-of-the-art detector without any robust classifier.
△ Less
Submitted 5 February, 2022;
originally announced February 2022.
-
Security Evaluation of Block-based Image Encryption for Vision Transformer against Jigsaw Puzzle Solver Attack
Authors:
Tatsuya Chuman,
Hitoshi Kiya
Abstract:
The aim of this paper is to evaluate the security of a block-based image encryption for the vision transformer against jigsaw puzzle solver attacks. The vision transformer, a model for image classification based on the transformer architecture, is carried out by dividing an image into a grid of square patches. Some encryption schemes for the vision transformer have been proposed by applying block-…
▽ More
The aim of this paper is to evaluate the security of a block-based image encryption for the vision transformer against jigsaw puzzle solver attacks. The vision transformer, a model for image classification based on the transformer architecture, is carried out by dividing an image into a grid of square patches. Some encryption schemes for the vision transformer have been proposed by applying block-based image encryption such as block scrambling and rotating to patches of the image. On the other hand, the security of encryption scheme for the vision transformer has never evaluated. In this paper, jigsaw puzzle solver attacks are utilized to evaluate the security of encrypted images by regarding the divided patches as pieces of a jigsaw puzzle. In experiments, an image is resized and divided into patches to apply block scrambling-based image encryption, and then the security of encrypted images for the vision transformer against jigsaw puzzle solver attacks is evaluated.
△ Less
Submitted 1 February, 2022;
originally announced February 2022.
-
A Privacy-Preserving Image Retrieval Scheme with a Mixture of Plain and EtC Images
Authors:
Kenta Iida,
Hitoshi Kiya
Abstract:
In this paper, we propose a novel content-based image-retrieval scheme that allows us to use a mixture of plain images and compressible encrypted ones called "encryption-then-compression (EtC) images." In the proposed scheme, extended SIMPLE descriptors are extracted from EtC images as well as from plain ones, so the mixed use of plain and encrypted images is available for image retrieval. In an e…
▽ More
In this paper, we propose a novel content-based image-retrieval scheme that allows us to use a mixture of plain images and compressible encrypted ones called "encryption-then-compression (EtC) images." In the proposed scheme, extended SIMPLE descriptors are extracted from EtC images as well as from plain ones, so the mixed use of plain and encrypted images is available for image retrieval. In an experiment, the proposed scheme was demonstrated to have almost the same retrieval performance as that for plain images, even with a mixture of plain and encrypted images.
△ Less
Submitted 1 February, 2022;
originally announced February 2022.
-
Access Control of Object Detection Models Using Encrypted Feature Maps
Authors:
Teru Nagamori,
Hiroki Ito,
April Pyone Maung Maung,
Hitoshi Kiya
Abstract:
In this paper, we propose an access control method for object detection models. The use of encrypted images or encrypted feature maps has been demonstrated to be effective in access control of models from unauthorized access. However, the effectiveness of the approach has been confirmed in only image classification models and semantic segmentation models, but not in object detection models. In thi…
▽ More
In this paper, we propose an access control method for object detection models. The use of encrypted images or encrypted feature maps has been demonstrated to be effective in access control of models from unauthorized access. However, the effectiveness of the approach has been confirmed in only image classification models and semantic segmentation models, but not in object detection models. In this paper, the use of encrypted feature maps is shown to be effective in access control of object detection models for the first time.
△ Less
Submitted 10 March, 2022; v1 submitted 1 February, 2022;
originally announced February 2022.
-
An Overview of Compressible and Learnable Image Transformation with Secret Key and Its Applications
Authors:
Hitoshi Kiya,
AprilPyone MaungMaung,
Yuma Kinoshita,
Shoko Imaizumi,
Sayaka Shiota
Abstract:
This article presents an overview of image transformation with a secret key and its applications. Image transformation with a secret key enables us not only to protect visual information on plain images but also to embed unique features controlled with a key into images. In addition, numerous encryption methods can generate encrypted images that are compressible and learnable for machine learning.…
▽ More
This article presents an overview of image transformation with a secret key and its applications. Image transformation with a secret key enables us not only to protect visual information on plain images but also to embed unique features controlled with a key into images. In addition, numerous encryption methods can generate encrypted images that are compressible and learnable for machine learning. Various applications of such transformation have been developed by using these properties. In this paper, we focus on a class of image transformation referred to as learnable image encryption, which is applicable to privacy-preserving machine learning and adversarially robust defense. Detailed descriptions of both transformation algorithms and performances are provided. Moreover, we discuss robustness against various attacks.
△ Less
Submitted 15 April, 2022; v1 submitted 26 January, 2022;
originally announced January 2022.
-
Proxy System with JPEG Bitstream-Based File-Size Preserving Encryption for Cloud Photo Streams
Authors:
Hiroyuki Kobayashi,
Hitoshi Kiya
Abstract:
In this paper, we propose a proxy system with JPEG bitstream-based file-size preserving encryption to securely store compressed images in cloud environments. The proposed system, which is settled between client's device and the Internet, allows us not only to have exact the same file size as that of original JPEG streams but also to maintain a predetermined image format. In an experiment, the prop…
▽ More
In this paper, we propose a proxy system with JPEG bitstream-based file-size preserving encryption to securely store compressed images in cloud environments. The proposed system, which is settled between client's device and the Internet, allows us not only to have exact the same file size as that of original JPEG streams but also to maintain a predetermined image format. In an experiment, the proposed system is verified to be effective in two cloud photo steams: Google Photo and iCloud Photo.
△ Less
Submitted 4 November, 2021;
originally announced January 2022.
-
Protection of SVM Model with Secret Key from Unauthorized Access
Authors:
Ryota Iijima,
AprilPyone MaungMaung,
Hitoshi Kiya
Abstract:
In this paper, we propose a block-wise image transformation method with a secret key for support vector machine (SVM) models. Models trained by using transformed images offer a poor performance to unauthorized users without a key, while they can offer a high performance to authorized users with a key. The proposed method is demonstrated to be robust enough against unauthorized access even under th…
▽ More
In this paper, we propose a block-wise image transformation method with a secret key for support vector machine (SVM) models. Models trained by using transformed images offer a poor performance to unauthorized users without a key, while they can offer a high performance to authorized users with a key. The proposed method is demonstrated to be robust enough against unauthorized access even under the use of kernel functions in a facial recognition experiment.
△ Less
Submitted 17 November, 2021;
originally announced November 2021.
-
Self-Supervised Intrinsic Image Decomposition Network Considering Reflectance Consistency
Authors:
Yuma Kinoshita,
Hitoshi Kiya
Abstract:
We propose a novel intrinsic image decomposition network considering reflectance consistency. Intrinsic image decomposition aims to decompose an image into illumination-invariant and illumination-variant components, referred to as ``reflectance'' and ``shading,'' respectively. Although there are three consistencies that the reflectance and shading should satisfy, most conventional work does not su…
▽ More
We propose a novel intrinsic image decomposition network considering reflectance consistency. Intrinsic image decomposition aims to decompose an image into illumination-invariant and illumination-variant components, referred to as ``reflectance'' and ``shading,'' respectively. Although there are three consistencies that the reflectance and shading should satisfy, most conventional work does not sufficiently account for consistency with respect to reflectance, owing to the use of a white-illuminant decomposition model and the lack of training images capturing the same objects under various illumination-brightness and -color conditions. For this reason, the three consistencies are considered in the proposed network by using a color-illuminant model and training the network with losses calculated from images taken under various illumination conditions. In addition, the proposed network can be trained in a self-supervised manner because various illumination conditions can easily be simulated. Experimental results show that our network can decompose images into reflectance and shading components.
△ Less
Submitted 5 November, 2021;
originally announced November 2021.
-
A Privacy-Preserving Image Retrieval Scheme Using A Codebook Generated From Independent Plain-Image Dataset
Authors:
Kenta Iida,
Hitoshi Kiya
Abstract:
In this paper, we propose a privacy-preserving image-retrieval scheme using a codebook generated by using a plain-image dataset. Encryption-then-compression (EtC) images, which were proposed for EtC systems, have been used in conventional privacy-preserving image-retrieval schemes, in which a codebook is generated from EtC images uploaded by image owners, and extended SIMPLE descriptors are then c…
▽ More
In this paper, we propose a privacy-preserving image-retrieval scheme using a codebook generated by using a plain-image dataset. Encryption-then-compression (EtC) images, which were proposed for EtC systems, have been used in conventional privacy-preserving image-retrieval schemes, in which a codebook is generated from EtC images uploaded by image owners, and extended SIMPLE descriptors are then calculated as image descriptors by using the codebook. In contrast, in the proposed scheme, a codebook is generated from a dataset independent of uploaded images. The use of an independent dataset enables us not only to use a codebook that does not require recalculation but also to constantly provide a high retrieval accuracy. In an experiment, the proposed scheme is demonstrated to maintain a high retrieval performance, even if codebooks are generated from a plain image dataset independent of image owners' encrypted images.
△ Less
Submitted 4 September, 2021;
originally announced September 2021.
-
Spatially varying white balancing for mixed and non-uniform illuminants
Authors:
Teruaki Akazawa,
Yuma Kinoshita,
Hitoshi Kiya
Abstract:
In this paper, we propose a novel white balance adjustment, called "spatially varying white balancing," for single, mixed, and non-uniform illuminants. By using n diagonal matrices along with a weight, the proposed method can reduce lighting effects on all spatially varying colors in an image under such illumination conditions. In contrast, conventional white balance adjustments do not consider th…
▽ More
In this paper, we propose a novel white balance adjustment, called "spatially varying white balancing," for single, mixed, and non-uniform illuminants. By using n diagonal matrices along with a weight, the proposed method can reduce lighting effects on all spatially varying colors in an image under such illumination conditions. In contrast, conventional white balance adjustments do not consider the correcting of all colors except under a single illuminant. Also, multi-color balance adjustments can map multiple colors into corresponding ground truth colors, although they may cause the rank deficiency problem to occur as a non-diagonal matrix is used, unlike white balancing. In an experiment, the effectiveness of the proposed method is shown under mixed and non-uniform illuminants, compared with conventional white and multi-color balancing. Moreover, under a single illuminant, the proposed method has almost the same performance as the conventional white balancing.
△ Less
Submitted 3 September, 2021;
originally announced September 2021.
