Showing 1–50 of 299 results for author: Uchida, S

Insights into the dependence of galaxy properties on the environment with explainable machine learning models

Authors: Shun-ya S. Uchida, Suchetha Cooray, Atsushi J. Nishizawa, Tsutomu T. Takeuchi, Peter Behroozi

Abstract: Galaxies reside within dark matter halos, but their properties are influenced not only by their halo properties but also by the surrounding environment. We construct an interpretable neural network framework to characterize the surrounding environment of galaxies and investigate the extent to which their properties are affected by neighboring galaxies in IllustrisTNG300 data ($z=0$). Our models pr… ▽ More Galaxies reside within dark matter halos, but their properties are influenced not only by their halo properties but also by the surrounding environment. We construct an interpretable neural network framework to characterize the surrounding environment of galaxies and investigate the extent to which their properties are affected by neighboring galaxies in IllustrisTNG300 data ($z=0$). Our models predict galaxy properties (stellar mass and star formation rate) given dark matter subhalo properties of both host subhalo and of surrounding galaxies, which serve as an explainable, flexible galaxy-halo connection model. We find that prediction accuracy peaks when incorporating only the nearest neighboring galaxy for stellar mass prediction, while star formation rate prediction benefits from information from up to the third-nearest neighbor. We determine that environmental influence follows a clear hierarchical pattern, with the nearest neighbor providing the dominant contribution that diminishes rapidly with additional neighbors. We confirm that central and satellite galaxies, as well as different galaxy categories based on mass and star-forming activity, exhibit distinct environmental dependencies. Environmental dependence for low-mass galaxies ($\log(M_*/M_\odot) < 10$) shows 35-50% environmental contribution compared to just 8-30% for massive centrals, while satellite galaxies experience consistently stronger environmental effects than centrals across all populations. Furthermore, we find that the most significant attribute from neighboring subhalos for predicting target galaxy properties is its distance to the nearest neighboring galaxy. These quantitative results offer guidance for constructing more sophisticated empirical and semi-analytic models of galaxy formation that explicitly include environmental dependence as a function of galaxy type and mass. △ Less

Submitted 10 May, 2025; originally announced May 2025.

Comments: submitted to MNRAS

Instance-wise Supervision-level Optimization in Active Learning

Authors: Shinnosuke Matsuo, Riku Togashi, Ryoma Bise, Seiichi Uchida, Masahiro Nomura

Abstract: Active learning (AL) is a label-efficient machine learning paradigm that focuses on selectively annotating high-value instances to maximize learning efficiency. Its effectiveness can be further enhanced by incorporating weak supervision, which uses rough yet cost-effective annotations instead of exact (i.e., full) but expensive annotations. We introduce a novel AL framework, Instance-wise Supervis… ▽ More Active learning (AL) is a label-efficient machine learning paradigm that focuses on selectively annotating high-value instances to maximize learning efficiency. Its effectiveness can be further enhanced by incorporating weak supervision, which uses rough yet cost-effective annotations instead of exact (i.e., full) but expensive annotations. We introduce a novel AL framework, Instance-wise Supervision-Level Optimization (ISO), which not only selects the instances to annotate but also determines their optimal annotation level within a fixed annotation budget. Its optimization criterion leverages the value-to-cost ratio (VCR) of each instance while ensuring diversity among the selected instances. In classification experiments, ISO consistently outperforms traditional AL methods and surpasses a state-of-the-art AL approach that combines full and weak supervision, achieving higher accuracy at a lower overall cost. This code is available at https://github.com/matsuo-shinnosuke/ISOAL. △ Less

Submitted 9 March, 2025; originally announced March 2025.

Comments: Accepted at CVPR2025

Proximity-Induced Nodal Metal in an Extremely Underdoped CuO$_2$ Plane in Triple-Layer Cuprates

Authors: Shin-ichiro Ideta, Shintaro Adachi, Takashi Noji, Shunpei Yamaguchi, Nae Sasaki, Shigeyuki Ishida, Shin-ichi Uchida, Takenori Fujii, Takao Watanabe, Wen O. Wang, Brian Moritz, Thomas P. Devereaux, Masashi Arita, Chung-Yu Mou, Teppei Yoshida, Kiyohisa Tanaka, Ting-Kuo Lee, Atsushi Fujimori

Abstract: ARPES studies have established that the high-$T_c$ cuprates with single and double CuO$_2$ layers evolve from the Mott insulator to the pseudogap state with a Fermi arc, on which the superconducting (SC) gap opens. In four- to six-layer cuprates, on the other hand, small hole Fermi pockets are formed in the innermost CuO$_2$ planes, indicating antiferromagnetism. Here, we performed ARPES studies o… ▽ More ARPES studies have established that the high-$T_c$ cuprates with single and double CuO$_2$ layers evolve from the Mott insulator to the pseudogap state with a Fermi arc, on which the superconducting (SC) gap opens. In four- to six-layer cuprates, on the other hand, small hole Fermi pockets are formed in the innermost CuO$_2$ planes, indicating antiferromagnetism. Here, we performed ARPES studies on the triple-layer Bi$_2$Sr$_2$Ca$_2$Cu$_3$O$_{10+δ}$ over a wide doping range, and found that, although the doping level of the inner CuO$_2$ plane was extremely low in underdoped samples, the $d$-wave SC gap was enhanced to the unprecedentedly large value of $Δ_0\sim$100 meV at the antinode and persisted well above $T_{c}$ without the appearance of a Fermi arc, indicating a robust ``nodal metal''. We attribute the nodal metallic behavior to the unique local environment of the inner clean CuO$_2$ plane in the triple-layer cuprates, sandwiched by nearly optimally-doped two outer CuO$_2$ planes and hence subject to strong proximity effect from both sides. In the nodal metal, quasiparticle peaks showed electron-hole symmetry, suggesting $d$-wave pairing fluctuations. Thus the proximity effect on the innermost CuO${_2}$ plane is the strongest in the triple-layer cuprates, which explains why the $T_c$ reaches the maximum at the layer number of three in every multi-layer cuprate family. △ Less

Submitted 21 February, 2025; originally announced February 2025.

Type-R: Automatically Retouching Typos for Text-to-Image Generation

Authors: Wataru Shimoda, Naoto Inoue, Daichi Haraguchi, Hayato Mitani, Seiichi Uchida, Kota Yamaguchi

Abstract: While recent text-to-image models can generate photorealistic images from text prompts that reflect detailed instructions, they still face significant challenges in accurately rendering words in the image. In this paper, we propose to retouch erroneous text renderings in the post-processing pipeline. Our approach, called Type-R, identifies typographical errors in the generated image, erases the er… ▽ More While recent text-to-image models can generate photorealistic images from text prompts that reflect detailed instructions, they still face significant challenges in accurately rendering words in the image. In this paper, we propose to retouch erroneous text renderings in the post-processing pipeline. Our approach, called Type-R, identifies typographical errors in the generated image, erases the erroneous text, regenerates text boxes for missing words, and finally corrects typos in the rendered words. Through extensive experiments, we show that Type-R, in combination with the latest text-to-image models such as Stable Diffusion or Flux, achieves the highest text rendering accuracy while maintaining image quality and also outperforms text-focused generation baselines in terms of balancing text accuracy and image quality. △ Less

Submitted 30 April, 2025; v1 submitted 27 November, 2024; originally announced November 2024.

Comments: Accepted to CVPR 2025. Codes: https://github.com/CyberAgentAILab/Type-R

Self-Relaxed Joint Training: Sample Selection for Severity Estimation with Ordinal Noisy Labels

Authors: Shumpei Takezaki, Kiyohito Tanaka, Seiichi Uchida

Abstract: Severity level estimation is a crucial task in medical image diagnosis. However, accurately assigning severity class labels to individual images is very costly and challenging. Consequently, the attached labels tend to be noisy. In this paper, we propose a new framework for training with ``ordinal'' noisy labels. Since severity levels have an ordinal relationship, we can leverage this to train a c… ▽ More Severity level estimation is a crucial task in medical image diagnosis. However, accurately assigning severity class labels to individual images is very costly and challenging. Consequently, the attached labels tend to be noisy. In this paper, we propose a new framework for training with ``ordinal'' noisy labels. Since severity levels have an ordinal relationship, we can leverage this to train a classifier while mitigating the negative effects of noisy labels. Our framework uses two techniques: clean sample selection and dual-network architecture. A technical highlight of our approach is the use of soft labels derived from noisy hard labels. By appropriately using the soft and hard labels in the two techniques, we achieve more accurate sample selection and robust network training. The proposed method outperforms various state-of-the-art methods in experiments using two endoscopic ulcerative colitis (UC) datasets and a retinal Diabetic Retinopathy (DR) dataset. Our codes are available at https://github.com/shumpei-takezaki/Self-Relaxed-Joint-Training. △ Less

Submitted 29 October, 2024; originally announced October 2024.

Comments: Accepted at WACV2025

Can GPTs Evaluate Graphic Design Based on Design Principles?

Authors: Daichi Haraguchi, Naoto Inoue, Wataru Shimoda, Hayato Mitani, Seiichi Uchida, Kota Yamaguchi

Abstract: Recent advancements in foundation models show promising capability in graphic design generation. Several studies have started employing Large Multimodal Models (LMMs) to evaluate graphic designs, assuming that LMMs can properly assess their quality, but it is unclear if the evaluation is reliable. One way to evaluate the quality of graphic design is to assess whether the design adheres to fundamen… ▽ More Recent advancements in foundation models show promising capability in graphic design generation. Several studies have started employing Large Multimodal Models (LMMs) to evaluate graphic designs, assuming that LMMs can properly assess their quality, but it is unclear if the evaluation is reliable. One way to evaluate the quality of graphic design is to assess whether the design adheres to fundamental graphic design principles, which are the designer's common practice. In this paper, we compare the behavior of GPT-based evaluation and heuristic evaluation based on design principles using human annotations collected from 60 subjects. Our experiments reveal that, while GPTs cannot distinguish small details, they have a reasonably good correlation with human annotation and exhibit a similar tendency to heuristic metrics based on design principles, suggesting that they are indeed capable of assessing the quality of graphic design. Our dataset is available at https://cyberagentailab.github.io/Graphic-design-evaluation . △ Less

Submitted 11 October, 2024; originally announced October 2024.

Comments: Accepted to SIGGRAPH Asia 2024 (Technical Communications Track)

Singularities in bivariate normal mixtures

Authors: Yutaro Kabata, Hirotaka Matsumoto, Seiichi Uchida, Masao Ueki

Abstract: We investigate mappings $F = (f_1, f_2) \colon \mathbb{R}^2 \to \mathbb{R}^2 $ where $ f_1, f_2 $ are bivariate normal densities from the perspective of singularity theory of mappings, motivated by the need to understand properties of two-component bivariate normal mixtures. We show a classification of mappings $ F = (f_1, f_2) $ via $\mathcal{A}$-equivalence and characterize them using statistica… ▽ More We investigate mappings $F = (f_1, f_2) \colon \mathbb{R}^2 \to \mathbb{R}^2 $ where $ f_1, f_2 $ are bivariate normal densities from the perspective of singularity theory of mappings, motivated by the need to understand properties of two-component bivariate normal mixtures. We show a classification of mappings $ F = (f_1, f_2) $ via $\mathcal{A}$-equivalence and characterize them using statistical notions. Our analysis reveals three distinct types, each with specific geometric properties. Furthermore, we determine the upper bounds for the number of modes in the mixture for each type. △ Less

Submitted 9 March, 2025; v1 submitted 1 October, 2024; originally announced October 2024.

Comments: 12 page, 5 figures. We revised Section 4 to correct an error in the description; note that the main results remain unchanged

Deep Bayesian Active Learning-to-Rank with Relative Annotation for Estimation of Ulcerative Colitis Severity

Authors: Takeaki Kadota, Hideaki Hayashi, Ryoma Bise, Kiyohito Tanaka, Seiichi Uchida

Abstract: Automatic image-based severity estimation is an important task in computer-aided diagnosis. Severity estimation by deep learning requires a large amount of training data to achieve a high performance. In general, severity estimation uses training data annotated with discrete (i.e., quantized) severity labels. Annotating discrete labels is often difficult in images with ambiguous severity, and the… ▽ More Automatic image-based severity estimation is an important task in computer-aided diagnosis. Severity estimation by deep learning requires a large amount of training data to achieve a high performance. In general, severity estimation uses training data annotated with discrete (i.e., quantized) severity labels. Annotating discrete labels is often difficult in images with ambiguous severity, and the annotation cost is high. In contrast, relative annotation, in which the severity between a pair of images is compared, can avoid quantizing severity and thus makes it easier. We can estimate relative disease severity using a learning-to-rank framework with relative annotations, but relative annotation has the problem of the enormous number of pairs that can be annotated. Therefore, the selection of appropriate pairs is essential for relative annotation. In this paper, we propose a deep Bayesian active learning-to-rank that automatically selects appropriate pairs for relative annotation. Our method preferentially annotates unlabeled pairs with high learning efficiency from the model uncertainty of the samples. We prove the theoretical basis for adapting Bayesian neural networks to pairwise learning-to-rank and demonstrate the efficiency of our method through experiments on endoscopic images of ulcerative colitis on both private and public datasets. We also show that our method achieves a high performance under conditions of significant class imbalance because it automatically selects samples from the minority classes. △ Less

Submitted 9 September, 2024; v1 submitted 7 September, 2024; originally announced September 2024.

Comments: 14 pages, 8 figures, accepted in Medical Image Analysis 2024

Journal ref: Medical Image Analysis 2024

Optimal control problem of evolution equation governed by hypergraph Laplacian

Authors: Takeshi Fukao, Masahiro Ikeda, Shun Uchida

Abstract: In this paper, we consider an optimal control problem of an ordinary differential inclusion governed by the hypergraph Laplacian, which is defined as a subdifferential of a convex function and then is a set-valued operator. We can assure the existence of optimal control for a suitable cost function by using methods of a priori estimates established in the previous studies. However, due to the mult… ▽ More In this paper, we consider an optimal control problem of an ordinary differential inclusion governed by the hypergraph Laplacian, which is defined as a subdifferential of a convex function and then is a set-valued operator. We can assure the existence of optimal control for a suitable cost function by using methods of a priori estimates established in the previous studies. However, due to the multivaluedness of the hypergraph Laplacian, it seems to be difficult to derive the necessary optimality condition for this problem. To cope with this difficulty, we introduce an approximation operator based on the approximation method of the hypergraph, so-called ``clique expansion.'' We first consider the optimality condition of the approximation problem with the clique expansion of the hypergraph Laplacian and next discuss the convergence to the original problem. In appendix, we state some basic properties of the clique expansion of the hypergraph Laplacian for future works. △ Less

Submitted 31 August, 2024; originally announced September 2024.

Comments: 37 pages, 4 figures

MSC Class: 49K15 (Primary) 05C65; 34G25; 49J15; 49J53 (Secondary)

Learning from Partial Label Proportions for Whole Slide Image Segmentation

Authors: Shinnosuke Matsuo, Daiki Suehiro, Seiichi Uchida, Hiroaki Ito, Kazuhiro Terada, Akihiko Yoshizawa, Ryoma Bise

Abstract: In this paper, we address the segmentation of tumor subtypes in whole slide images (WSI) by utilizing incomplete label proportions. Specifically, we utilize `partial' label proportions, which give the proportions among tumor subtypes but do not give the proportion between tumor and non-tumor. Partial label proportions are recorded as the standard diagnostic information by pathologists, and we, the… ▽ More In this paper, we address the segmentation of tumor subtypes in whole slide images (WSI) by utilizing incomplete label proportions. Specifically, we utilize `partial' label proportions, which give the proportions among tumor subtypes but do not give the proportion between tumor and non-tumor. Partial label proportions are recorded as the standard diagnostic information by pathologists, and we, therefore, want to use them for realizing the segmentation model that can classify each WSI patch into one of the tumor subtypes or non-tumor. We call this problem ``learning from partial label proportions (LPLP)'' and formulate the problem as a weakly supervised learning problem. Then, we propose an efficient algorithm for this challenging problem by decomposing it into two weakly supervised learning subproblems: multiple instance learning (MIL) and learning from label proportions (LLP). These subproblems are optimized efficiently in the end-to-end manner. The effectiveness of our algorithm is demonstrated through experiments conducted on two WSI datasets. △ Less

Submitted 14 May, 2024; originally announced May 2024.

Comments: Accepted at MICCAI2024

Test-Time Augmentation for Traveling Salesperson Problem

Authors: Ryo Ishiyama, Takahiro Shirakawa, Seiichi Uchida, Shinnosuke Matsuo

Abstract: We propose Test-Time Augmentation (TTA) as an effective technique for addressing combinatorial optimization problems, including the Traveling Salesperson Problem. In general, deep learning models possessing the property of invariance, where the output is uniquely determined regardless of the node indices, have been proposed to learn graph structures efficiently. In contrast, we interpret the permu… ▽ More We propose Test-Time Augmentation (TTA) as an effective technique for addressing combinatorial optimization problems, including the Traveling Salesperson Problem. In general, deep learning models possessing the property of invariance, where the output is uniquely determined regardless of the node indices, have been proposed to learn graph structures efficiently. In contrast, we interpret the permutation of node indices, which exchanges the elements of the distance matrix, as a TTA scheme. The results demonstrate that our method is capable of obtaining shorter solutions than the latest models. Furthermore, we show that the probability of finding a solution closer to an exact solution increases depending on the augmentation size. △ Less

Submitted 7 May, 2024; originally announced May 2024.

Pseudo-label Learning with Calibrated Confidence Using an Energy-based Model

Authors: Masahito Toba, Seiichi Uchida, Hideaki Hayashi

Abstract: In pseudo-labeling (PL), which is a type of semi-supervised learning, pseudo-labels are assigned based on the confidence scores provided by the classifier; therefore, accurate confidence is important for successful PL. In this study, we propose a PL algorithm based on an energy-based model (EBM), which is referred to as the energy-based PL (EBPL). In EBPL, a neural network-based classifier and an… ▽ More In pseudo-labeling (PL), which is a type of semi-supervised learning, pseudo-labels are assigned based on the confidence scores provided by the classifier; therefore, accurate confidence is important for successful PL. In this study, we propose a PL algorithm based on an energy-based model (EBM), which is referred to as the energy-based PL (EBPL). In EBPL, a neural network-based classifier and an EBM are jointly trained by sharing their feature extraction parts. This approach enables the model to learn both the class decision boundary and input data distribution, enhancing confidence calibration during network training. The experimental results demonstrate that EBPL outperforms the existing PL method in semi-supervised image classification tasks, with superior confidence calibration error and recognition accuracy. △ Less

Submitted 15 April, 2024; originally announced April 2024.

Comments: 8 pages, 8 figures, Accepted at IJCNN 2024

Total Disentanglement of Font Images into Style and Character Class Features

Authors: Daichi Haraguchi, Wataru Shimoda, Kota Yamaguchi, Seiichi Uchida

Abstract: In this paper, we demonstrate a total disentanglement of font images. Total disentanglement is a neural network-based method for decomposing each font image nonlinearly and completely into its style and content (i.e., character class) features. It uses a simple but careful training procedure to extract the common style feature from all `A'-`Z' images in the same font and the common content feature… ▽ More In this paper, we demonstrate a total disentanglement of font images. Total disentanglement is a neural network-based method for decomposing each font image nonlinearly and completely into its style and content (i.e., character class) features. It uses a simple but careful training procedure to extract the common style feature from all `A'-`Z' images in the same font and the common content feature from all `A' (or another class) images in different fonts. These disentangled features guarantee the reconstruction of the original font image. Various experiments have been conducted to understand the performance of total disentanglement. First, it is demonstrated that total disentanglement is achievable with very high accuracy; this is experimental proof of the long-standing open question, ``Does `A'-ness exist?'' Hofstadter (1985). Second, it is demonstrated that the disentangled features produced by total disentanglement apply to a variety of tasks, including font recognition, character recognition, and one-shot font image generation. △ Less

Submitted 19 March, 2024; originally announced March 2024.

NoiseCollage: A Layout-Aware Text-to-Image Diffusion Model Based on Noise Cropping and Merging

Authors: Takahiro Shirakawa, Seiichi Uchida

Abstract: Layout-aware text-to-image generation is a task to generate multi-object images that reflect layout conditions in addition to text conditions. The current layout-aware text-to-image diffusion models still have several issues, including mismatches between the text and layout conditions and quality degradation of generated images. This paper proposes a novel layout-aware text-to-image diffusion mode… ▽ More Layout-aware text-to-image generation is a task to generate multi-object images that reflect layout conditions in addition to text conditions. The current layout-aware text-to-image diffusion models still have several issues, including mismatches between the text and layout conditions and quality degradation of generated images. This paper proposes a novel layout-aware text-to-image diffusion model called NoiseCollage to tackle these issues. During the denoising process, NoiseCollage independently estimates noises for individual objects and then crops and merges them into a single noise. This operation helps avoid condition mismatches; in other words, it can put the right objects in the right places. Qualitative and quantitative evaluations show that NoiseCollage outperforms several state-of-the-art models. These successful results indicate that the crop-and-merge operation of noises is a reasonable strategy to control image generation. We also show that NoiseCollage can be integrated with ControlNet to use edges, sketches, and pose skeletons as additional conditions. Experimental results show that this integration boosts the layout accuracy of ControlNet. The code is available at https://github.com/univ-esuty/noisecollage. △ Less

Submitted 6 March, 2024; originally announced March 2024.

Comments: Accepted at CVPR 2024

Cross-Domain Image Conversion by CycleDM

Authors: Sho Shimotsumagari, Shumpei Takezaki, Daichi Haraguchi, Seiichi Uchida

Abstract: The purpose of this paper is to enable the conversion between machine-printed character images (i.e., font images) and handwritten character images through machine learning. For this purpose, we propose a novel unpaired image-to-image domain conversion method, CycleDM, which incorporates the concept of CycleGAN into the diffusion model. Specifically, CycleDM has two internal conversion models that… ▽ More The purpose of this paper is to enable the conversion between machine-printed character images (i.e., font images) and handwritten character images through machine learning. For this purpose, we propose a novel unpaired image-to-image domain conversion method, CycleDM, which incorporates the concept of CycleGAN into the diffusion model. Specifically, CycleDM has two internal conversion models that bridge the denoising processes of two image domains. These conversion models are efficiently trained without explicit correspondence between the domains. By applying machine-printed and handwritten character images to the two modalities, CycleDM realizes the conversion between them. Our experiments for evaluating the converted images quantitatively and qualitatively found that ours performs better than other comparable approaches. △ Less

Submitted 5 March, 2024; originally announced March 2024.

An Ordinal Diffusion Model for Generating Medical Images with Different Severity Levels

Authors: Shumpei Takezaki, Seiichi Uchida

Abstract: Diffusion models have recently been used for medical image generation because of their high image quality. In this study, we focus on generating medical images with ordinal classes, which have ordinal relationships, such as severity levels. We propose an Ordinal Diffusion Model (ODM) that controls the ordinal relationships of the estimated noise images among the classes. Our model was evaluated ex… ▽ More Diffusion models have recently been used for medical image generation because of their high image quality. In this study, we focus on generating medical images with ordinal classes, which have ordinal relationships, such as severity levels. We propose an Ordinal Diffusion Model (ODM) that controls the ordinal relationships of the estimated noise images among the classes. Our model was evaluated experimentally by generating retinal and endoscopic images of multiple severity classes. ODM achieved higher performance than conventional generative models by generating realistic images, especially in high-severity classes with fewer training samples. △ Less

Submitted 10 October, 2024; v1 submitted 1 March, 2024; originally announced March 2024.

Comments: Accepted at ISBI2024

What Text Design Characterizes Book Genres?

Authors: Daichi Haraguchi, Brian Kenji Iwana, Seiichi Uchida

Abstract: This study analyzes the relationship between non-verbal information (e.g., genres) and text design (e.g., font style, character color, etc.) through the classification of book genres using text design on book covers. Text images have both semantic information about the word itself and other information (non-semantic information or visual design), such as font style, character color, etc. When we r… ▽ More This study analyzes the relationship between non-verbal information (e.g., genres) and text design (e.g., font style, character color, etc.) through the classification of book genres using text design on book covers. Text images have both semantic information about the word itself and other information (non-semantic information or visual design), such as font style, character color, etc. When we read a word printed on some materials, we receive impressions or other information from both the word itself and the visual design. Basically, we can understand verbal information only from semantic information, i.e., the words themselves; however, we can consider that text design is helpful for understanding other additional information (i.e., non-verbal information), such as impressions, genre, etc. To investigate the effect of text design, we analyze text design using words printed on book covers and their genres in two scenarios. First, we attempted to understand the importance of visual design for determining the genre (i.e., non-verbal information) of books by analyzing the differences in the relationship between semantic information/visual design and genres. In the experiment, we found that semantic information is sufficient to determine the genre; however, text design is helpful in adding more discriminative features for book genres. Second, we investigated the effect of each text design on book genres. As a result, we found that each text design characterizes some book genres. For example, font style is useful to add more discriminative features for genres of ``Mystery, Thriller \& Suspense'' and ``Christian books \& Bibles.'' △ Less

Submitted 26 February, 2024; originally announced February 2024.

Impression-CLIP: Contrastive Shape-Impression Embedding for Fonts

Authors: Yugo Kubota, Daichi Haraguchi, Seiichi Uchida

Abstract: Fonts convey different impressions to readers. These impressions often come from the font shapes. However, the correlation between fonts and their impression is weak and unstable because impressions are subjective. To capture such weak and unstable cross-modal correlation between font shapes and their impressions, we propose Impression-CLIP, which is a novel machine-learning model based on CLIP (C… ▽ More Fonts convey different impressions to readers. These impressions often come from the font shapes. However, the correlation between fonts and their impression is weak and unstable because impressions are subjective. To capture such weak and unstable cross-modal correlation between font shapes and their impressions, we propose Impression-CLIP, which is a novel machine-learning model based on CLIP (Contrastive Language-Image Pre-training). By using the CLIP-based model, font image features and their impression features are pulled closer, and font image features and unrelated impression features are pushed apart. This procedure realizes co-embedding between font image and their impressions. In our experiment, we perform cross-modal retrieval between fonts and impressions through co-embedding. The results indicate that Impression-CLIP achieves better retrieval accuracy than the state-of-the-art method. Additionally, our model shows the robustness to noise and missing tags. △ Less

Submitted 26 February, 2024; originally announced February 2024.

Font Impression Estimation in the Wild

Authors: Kazuki Kitajima, Daichi Haraguchi, Seiichi Uchida

Abstract: This paper addresses the challenging task of estimating font impressions from real font images. We use a font dataset with annotation about font impressions and a convolutional neural network (CNN) framework for this task. However, impressions attached to individual fonts are often missing and noisy because of the subjective characteristic of font impression annotation. To realize stable impressio… ▽ More This paper addresses the challenging task of estimating font impressions from real font images. We use a font dataset with annotation about font impressions and a convolutional neural network (CNN) framework for this task. However, impressions attached to individual fonts are often missing and noisy because of the subjective characteristic of font impression annotation. To realize stable impression estimation even with such a dataset, we propose an exemplar-based impression estimation approach, which relies on a strategy of ensembling impressions of exemplar fonts that are similar to the input image. In addition, we train CNN with synthetic font images that mimic scanned word images so that CNN estimates impressions of font images in the wild. We evaluate the basic performance of the proposed estimation method quantitatively and qualitatively. Then, we conduct a correlation analysis between book genres and font impressions on real book cover images; it is important to note that this analysis is only possible with our impression estimation method. The analysis reveals various trends in the correlation between them - this fact supports a hypothesis that book cover designers carefully choose a font for a book cover considering the impression given by the font. △ Less

Submitted 23 February, 2024; originally announced February 2024.

Typographic Text Generation with Off-the-Shelf Diffusion Model

Authors: KhayTze Peong, Seiichi Uchida, Daichi Haraguchi

Abstract: Recent diffusion-based generative models show promise in their ability to generate text images, but limitations in specifying the styles of the generated texts render them insufficient in the realm of typographic design. This paper proposes a typographic text generation system to add and modify text on typographic designs while specifying font styles, colors, and text effects. The proposed system… ▽ More Recent diffusion-based generative models show promise in their ability to generate text images, but limitations in specifying the styles of the generated texts render them insufficient in the realm of typographic design. This paper proposes a typographic text generation system to add and modify text on typographic designs while specifying font styles, colors, and text effects. The proposed system is a novel combination of two off-the-shelf methods for diffusion models, ControlNet and Blended Latent Diffusion. The former functions to generate text images under the guidance of edge conditions specifying stroke contours. The latter blends latent noise in Latent Diffusion Models (LDM) to add typographic text naturally onto an existing background. We first show that given appropriate text edges, ControlNet can generate texts in specified fonts while incorporating effects described by prompts. We further introduce text edge manipulation as an intuitive and customizable way to produce texts with complex effects such as ``shadows'' and ``reflections''. Finally, with the proposed system, we successfully add and modify texts on a predefined background while preserving its overall coherence. △ Less

Submitted 22 February, 2024; originally announced February 2024.

Learning to Kern: Set-wise Estimation of Optimal Letter Space

Authors: Kei Nakatsuru, Seiichi Uchida

Abstract: Kerning is the task of setting appropriate horizontal spaces for all possible letter pairs of a certain font. One of the difficulties of kerning is that the appropriate space differs for each letter pair. Therefore, for a total of 52 capital and small letters, we need to adjust $52 \times 52 = 2704$ different spaces. Another difficulty is that there is neither a general procedure nor criterion for… ▽ More Kerning is the task of setting appropriate horizontal spaces for all possible letter pairs of a certain font. One of the difficulties of kerning is that the appropriate space differs for each letter pair. Therefore, for a total of 52 capital and small letters, we need to adjust $52 \times 52 = 2704$ different spaces. Another difficulty is that there is neither a general procedure nor criterion for automatic kerning; therefore, kerning is still done manually or with heuristics. In this paper, we tackle kerning by proposing two machine-learning models, called pairwise and set-wise models. The former is a simple deep neural network that estimates the letter space for two given letter images. In contrast, the latter is a transformer-based model that estimates the letter spaces for three or more given letter images. For example, the set-wise model simultaneously estimates 2704 spaces for 52 letter images for a certain font. Among the two models, the set-wise model is not only more efficient but also more accurate because its internal self-attention mechanism allows for more consistent kerning for all letters. Experimental results on about 2500 Google fonts and their quantitative and qualitative analyses show that the set-wise model has an average estimation error of only about 5.3 pixels when the average letter space of all fonts and letter pairs is about 115 pixels. △ Less

Submitted 28 April, 2024; v1 submitted 22 February, 2024; originally announced February 2024.

Font Style Interpolation with Diffusion Models

Authors: Tetta Kondo, Shumpei Takezaki, Daichi Haraguchi, Seiichi Uchida

Abstract: Fonts have huge variations in their styles and give readers different impressions. Therefore, generating new fonts is worthy of giving new impressions to readers. In this paper, we employ diffusion models to generate new font styles by interpolating a pair of reference fonts with different styles. More specifically, we propose three different interpolation approaches, image-blending, condition-ble… ▽ More Fonts have huge variations in their styles and give readers different impressions. Therefore, generating new fonts is worthy of giving new impressions to readers. In this paper, we employ diffusion models to generate new font styles by interpolating a pair of reference fonts with different styles. More specifically, we propose three different interpolation approaches, image-blending, condition-blending, and noise-blending, with the diffusion models. We perform qualitative and quantitative experimental analyses to understand the style generation ability of the three approaches. According to experimental results, three proposed approaches can generate not only expected font styles but also somewhat serendipitous font styles. We also compare the approaches with a state-of-the-art style-conditional Latin-font generative network model to confirm the validity of using the diffusion models for the style interpolation task. △ Less

Submitted 22 February, 2024; originally announced February 2024.

Boosting for Bounding the Worst-class Error

Authors: Yuya Saito, Shinnosuke Matsuo, Seiichi Uchida, Daiki Suehiro

Abstract: This paper tackles the problem of the worst-class error rate, instead of the standard error rate averaged over all classes. For example, a three-class classification task with class-wise error rates of 10\%, 10\%, and 40\% has a worst-class error rate of 40\%, whereas the average is 20\% under the class-balanced condition. The worst-class error is important in many applications. For example, in a… ▽ More This paper tackles the problem of the worst-class error rate, instead of the standard error rate averaged over all classes. For example, a three-class classification task with class-wise error rates of 10\%, 10\%, and 40\% has a worst-class error rate of 40\%, whereas the average is 20\% under the class-balanced condition. The worst-class error is important in many applications. For example, in a medical image classification task, it would not be acceptable for the malignant tumor class to have a 40\% error rate, while the benign and healthy classes have 10\% error rates.We propose a boosting algorithm that guarantees an upper bound of the worst-class training error and derive its generalization bound. Experimental results show that the algorithm lowers worst-class test error rates while avoiding overfitting to the training set. △ Less

Submitted 20 October, 2023; originally announced October 2023.

Local Style Awareness of Font Images

Authors: Daichi Haraguchi, Seiichi Uchida

Abstract: When we compare fonts, we often pay attention to styles of local parts, such as serifs and curvatures. This paper proposes an attention mechanism to find important local parts. The local parts with larger attention are then considered important. The proposed mechanism can be trained in a quasi-self-supervised manner that requires no manual annotation other than knowing that a set of character imag… ▽ More When we compare fonts, we often pay attention to styles of local parts, such as serifs and curvatures. This paper proposes an attention mechanism to find important local parts. The local parts with larger attention are then considered important. The proposed mechanism can be trained in a quasi-self-supervised manner that requires no manual annotation other than knowing that a set of character images is from the same font, such as Helvetica. After confirming that the trained attention mechanism can find style-relevant local parts, we utilize the resulting attention for local style-aware font generation. Specifically, we design a new reconstruction loss function to put more weight on the local parts with larger attention for generating character images with more accurate style realization. This loss function has the merit of applicability to various font generation models. Our experimental results show that the proposed loss function improves the quality of generated character images by several few-shot font generation models. △ Less

Submitted 10 October, 2023; originally announced October 2023.

Comments: Accepted at ICDAR WML 2023

Deep Attentive Time Warping

Authors: Shinnosuke Matsuo, Xiaomeng Wu, Gantugs Atarsaikhan, Akisato Kimura, Kunio Kashino, Brian Kenji Iwana, Seiichi Uchida

Abstract: Similarity measures for time series are important problems for time series classification. To handle the nonlinear time distortions, Dynamic Time Warping (DTW) has been widely used. However, DTW is not learnable and suffers from a trade-off between robustness against time distortion and discriminative power. In this paper, we propose a neural network model for task-adaptive time warping. Specifica… ▽ More Similarity measures for time series are important problems for time series classification. To handle the nonlinear time distortions, Dynamic Time Warping (DTW) has been widely used. However, DTW is not learnable and suffers from a trade-off between robustness against time distortion and discriminative power. In this paper, we propose a neural network model for task-adaptive time warping. Specifically, we use the attention model, called the bipartite attention model, to develop an explicit time warping mechanism with greater distortion invariance. Unlike other learnable models using DTW for warping, our model predicts all local correspondences between two time series and is trained based on metric learning, which enables it to learn the optimal data-dependent warping for the target task. We also propose to induce pre-training of our model by DTW to improve the discriminative power. Extensive experiments demonstrate the superior effectiveness of our model over DTW and its state-of-the-art performance in online signature verification. △ Less

Submitted 13 September, 2023; originally announced September 2023.

Comments: Accepted at Pattern Recognition

Towards Diverse and Consistent Typography Generation

Authors: Wataru Shimoda, Daichi Haraguchi, Seiichi Uchida, Kota Yamaguchi

Abstract: In this work, we consider the typography generation task that aims at producing diverse typographic styling for the given graphic document. We formulate typography generation as a fine-grained attribute generation for multiple text elements and build an autoregressive model to generate diverse typography that matches the input design context. We further propose a simple yet effective sampling appr… ▽ More In this work, we consider the typography generation task that aims at producing diverse typographic styling for the given graphic document. We formulate typography generation as a fine-grained attribute generation for multiple text elements and build an autoregressive model to generate diverse typography that matches the input design context. We further propose a simple yet effective sampling approach that respects the consistency and distinction principle of typography so that generated examples share consistent typographic styling across text elements. Our empirical study shows that our model successfully generates diverse typographic designs while preserving a consistent typographic structure. △ Less

Submitted 5 September, 2023; originally announced September 2023.

Toward Defensive Letter Design

Authors: Rentaro Kataoka, Akisato Kimura, Seiichi Uchida

Abstract: A major approach for defending against adversarial attacks aims at controlling only image classifiers to be more resilient, and it does not care about visual objects, such as pandas and cars, in images. This means that visual objects themselves cannot take any defensive actions, and they are still vulnerable to adversarial attacks. In contrast, letters are artificial symbols, and we can freely con… ▽ More A major approach for defending against adversarial attacks aims at controlling only image classifiers to be more resilient, and it does not care about visual objects, such as pandas and cars, in images. This means that visual objects themselves cannot take any defensive actions, and they are still vulnerable to adversarial attacks. In contrast, letters are artificial symbols, and we can freely control their appearance unless losing their readability. In other words, we can make the letters more defensive to the attacks. This paper poses three research questions related to the adversarial vulnerability of letter images: (1) How defensive are the letters against adversarial attacks? (2) Can we estimate how defensive a given letter image is before attacks? (3) Can we control the letter images to be more defensive against adversarial attacks? For answering the first and second questions, we measure the defensibility of letters by employing Iterative Fast Gradient Sign Method (I-FGSM) and then build a deep regression model for estimating the defensibility of each letter image. We also propose a two-step method based on a generative adversarial network (GAN) for generating character images with higher defensibility, which solves the third research question. △ Less

Submitted 4 September, 2023; originally announced September 2023.

Comments: 14 pages, 8 figures, accepted at ACPR 2023

Selective Scene Text Removal

Authors: Hayato Mitani, Akisato Kimura, Seiichi Uchida

Abstract: Scene text removal (STR) is the image transformation task to remove text regions in scene images. The conventional STR methods remove all scene text. This means that the existing methods cannot select text to be removed. In this paper, we propose a novel task setting named selective scene text removal (SSTR) that removes only target words specified by the user. Although SSTR is a more complex task… ▽ More Scene text removal (STR) is the image transformation task to remove text regions in scene images. The conventional STR methods remove all scene text. This means that the existing methods cannot select text to be removed. In this paper, we propose a novel task setting named selective scene text removal (SSTR) that removes only target words specified by the user. Although SSTR is a more complex task than STR, the proposed multi-module structure enables efficient training for SSTR. Experimental results show that the proposed method can remove target words as expected. △ Less

Submitted 3 October, 2023; v1 submitted 1 September, 2023; originally announced September 2023.

Comments: 12 pages, 8 figures, Accepted at the 34th British Machine Vision Conference, code:https://github.com/mitanihayato/Selective-Scene-Text-Removal

Analyzing Font Style Usage and Contextual Factors in Real Images

Authors: Naoya Yasukochi, Hideaki Hayashi, Daichi Haraguchi, Seiichi Uchida

Abstract: There are various font styles in the world. Different styles give different impressions and readability. This paper analyzes the relationship between font styles and contextual factors that might affect font style selection with large-scale datasets. For example, we will analyze the relationship between font style and its surrounding object (such as ``bus'') by using about 800,000 words in the Ope… ▽ More There are various font styles in the world. Different styles give different impressions and readability. This paper analyzes the relationship between font styles and contextual factors that might affect font style selection with large-scale datasets. For example, we will analyze the relationship between font style and its surrounding object (such as ``bus'') by using about 800,000 words in the Open Images dataset. We also use a book cover dataset to analyze the relationship between font styles with book genres. Moreover, the meaning of the word is assumed as another contextual factor. For these numeric analyses, we utilize our own font-style feature extraction model and word2vec. As a result of co-occurrence-based relationship analysis, we found several instances of specific font styles being used for specific contextual factors. △ Less

Submitted 21 June, 2023; originally announced June 2023.

Comments: Accepted at ICDAR 2023

Ambigram Generation by A Diffusion Model

Authors: Takahiro Shirakawa, Seiichi Uchida

Abstract: Ambigrams are graphical letter designs that can be read not only from the original direction but also from a rotated direction (especially with 180 degrees). Designing ambigrams is difficult even for human experts because keeping their dual readability from both directions is often difficult. This paper proposes an ambigram generation model. As its generation module, we use a diffusion model, whic… ▽ More Ambigrams are graphical letter designs that can be read not only from the original direction but also from a rotated direction (especially with 180 degrees). Designing ambigrams is difficult even for human experts because keeping their dual readability from both directions is often difficult. This paper proposes an ambigram generation model. As its generation module, we use a diffusion model, which has recently been used to generate high-quality photographic images. By specifying a pair of letter classes, such as 'A' and 'B', the proposed model generates various ambigram images which can be read as 'A' from the original direction and 'B' from a direction rotated 180 degrees. Quantitative and qualitative analyses of experimental results show that the proposed model can generate high-quality and diverse ambigrams. In addition, we define ambigramability, an objective measure of how easy it is to generate ambigrams for each letter pair. For example, the pair of 'A' and 'V' shows a high ambigramability (that is, it is easy to generate their ambigrams), and the pair of 'D' and 'K' shows a lower ambigramability. The ambigramability gives various hints of the ambigram generation not only for computers but also for human experts. The code can be found at (https://github.com/univ-esuty/ambifusion). △ Less

Submitted 21 June, 2023; originally announced June 2023.

Comments: Accepted at ICDAR 2023

FETNet: Feature Erasing and Transferring Network for Scene Text Removal

Authors: Guangtao Lyu, Kun Liu, Anna Zhu, Seiichi Uchida, Brian Kenji Iwana

Abstract: The scene text removal (STR) task aims to remove text regions and recover the background smoothly in images for private information protection. Most existing STR methods adopt encoder-decoder-based CNNs, with direct copies of the features in the skip connections. However, the encoded features contain both text texture and structure information. The insufficient utilization of text features hampers… ▽ More The scene text removal (STR) task aims to remove text regions and recover the background smoothly in images for private information protection. Most existing STR methods adopt encoder-decoder-based CNNs, with direct copies of the features in the skip connections. However, the encoded features contain both text texture and structure information. The insufficient utilization of text features hampers the performance of background reconstruction in text removal regions. To tackle these problems, we propose a novel Feature Erasing and Transferring (FET) mechanism to reconfigure the encoded features for STR in this paper. In FET, a Feature Erasing Module (FEM) is designed to erase text features. An attention module is responsible for generating the feature similarity guidance. The Feature Transferring Module (FTM) is introduced to transfer the corresponding features in different layers based on the attention guidance. With this mechanism, a one-stage, end-to-end trainable network called FETNet is constructed for scene text removal. In addition, to facilitate research on both scene text removal and segmentation tasks, we introduce a novel dataset, Flickr-ST, with multi-category annotations. A sufficient number of experiments and ablation studies are conducted on the public datasets and Flickr-ST. Our proposed method achieves state-of-the-art performance using most metrics, with remarkably higher quality scene text removal results. The source code of our work is available at: \href{https://github.com/GuangtaoLyu/FETNet}{https://github.com/GuangtaoLyu/FETNet. △ Less

Submitted 15 June, 2023; originally announced June 2023.

Comments: Accepted by Pattern Recognition 2023

Journal ref: Pattern Recognition 2023

Contour Completion by Transformers and Its Application to Vector Font Data

Authors: Yusuke Nagata, Brian Kenji Iwana, Seiichi Uchida

Abstract: In documents and graphics, contours are a popular format to describe specific shapes. For example, in the True Type Font (TTF) file format, contours describe vector outlines of typeface shapes. Each contour is often defined as a sequence of points. In this paper, we tackle the contour completion task. In this task, the input is a contour sequence with missing points, and the output is a generated… ▽ More In documents and graphics, contours are a popular format to describe specific shapes. For example, in the True Type Font (TTF) file format, contours describe vector outlines of typeface shapes. Each contour is often defined as a sequence of points. In this paper, we tackle the contour completion task. In this task, the input is a contour sequence with missing points, and the output is a generated completed contour. This task is more difficult than image completion because, for images, the missing pixels are indicated. Since there is no such indication in the contour completion task, we must solve the problem of missing part detection and completion simultaneously. We propose a Transformer-based method to solve this problem and show the results of the typeface contour completion. △ Less

Submitted 27 April, 2023; originally announced April 2023.

Comments: Accepted at ICDAR 2023

Functional Knowledge Transfer with Self-supervised Representation Learning

Authors: Prakash Chandra Chhipa, Muskaan Chopra, Gopal Mengi, Varun Gupta, Richa Upadhyay, Meenakshi Subhash Chippa, Kanjar De, Rajkumar Saini, Seiichi Uchida, Marcus Liwicki

Abstract: This work investigates the unexplored usability of self-supervised representation learning in the direction of functional knowledge transfer. In this work, functional knowledge transfer is achieved by joint optimization of self-supervised learning pseudo task and supervised learning task, improving supervised learning task performance. Recent progress in self-supervised learning uses a large volum… ▽ More This work investigates the unexplored usability of self-supervised representation learning in the direction of functional knowledge transfer. In this work, functional knowledge transfer is achieved by joint optimization of self-supervised learning pseudo task and supervised learning task, improving supervised learning task performance. Recent progress in self-supervised learning uses a large volume of data, which becomes a constraint for its applications on small-scale datasets. This work shares a simple yet effective joint training framework that reinforces human-supervised task learning by learning self-supervised representations just-in-time and vice versa. Experiments on three public datasets from different visual domains, Intel Image, CIFAR, and APTOS, reveal a consistent track of performance improvements on classification tasks during joint optimization. Qualitative analysis also supports the robustness of learnt representations. Source code and trained models are available on GitHub. △ Less

Submitted 10 July, 2023; v1 submitted 12 March, 2023; originally announced April 2023.

Comments: Accepted at IEEE International Conference on Image Processing (ICIP 2023)

Differentiated roles of Lifshitz transition on thermodynamics and superconductivity in La2-xSrxCuO4

Authors: Yong Zhong, Zhuoyu Chen, Su-Di Chen, Ke-Jun Xu, Makoto Hashimoto, Yu He, Shin-ichi Uchida, Donghui Lu, Sung-Kwan Mo, Zhi-Xun Shen

Abstract: The effect of Lifshitz transition on thermodynamics and superconductivity in hole-doped cuprates has been heavily debated but remains an open question. In particular, an observed peak of electronic specific heat is proposed to originate from fluctuations of a putative quantum critical point p* (e.g. the termination of pseudogap at zero temperature), which is close to, but distinguishable from the… ▽ More The effect of Lifshitz transition on thermodynamics and superconductivity in hole-doped cuprates has been heavily debated but remains an open question. In particular, an observed peak of electronic specific heat is proposed to originate from fluctuations of a putative quantum critical point p* (e.g. the termination of pseudogap at zero temperature), which is close to, but distinguishable from the Lifshitz transition in La-based cuprates. Here, we report an in situ angle-resolved photoemission spectroscopy study of three-dimensional Fermi surfaces in La2-xSrxCuO4 thin films(x = 0.06 - 0.35). With accurate kz dispersion quantification, the Lifshitz transition is determined to happen within a finite range around x = 0.21. Normal state electronic specific heat, calculated from spectroscopy-derived band parameters, agrees with previous thermodynamic microcalorimetry measurements. The account of the specific heat maximum by underlying band structures excludes the need for additionally dominant contribution from the quantum fluctuations at p*. A d-wave superconducting gap smoothly across the Lifshitz transition demonstrates the insensitivity of superconductivity to the dramatic density of states enhancement. △ Less

Submitted 21 March, 2023; originally announced March 2023.

Journal ref: Proc. Natl. Acad. Sci. U.S.A. 119, e2204630119 (2022)

The evolution of cooperation and diversity by integrated indirect reciprocity

Authors: Tatsuya Sasaki, Satoshi Uchida, Isamu Okada, Hitoshi Yamamoto

Abstract: Indirect reciprocity is one of the major mechanisms for the evolution of cooperation in human societies. There are two types of indirect reciprocity: upstream and downstream. Cooperation in downstream reciprocity follows the pattern, 'You helped someone, and I will help you'. The direction of cooperation is reversed in upstream reciprocity, which instead follows the pattern, 'You helped me, and I… ▽ More Indirect reciprocity is one of the major mechanisms for the evolution of cooperation in human societies. There are two types of indirect reciprocity: upstream and downstream. Cooperation in downstream reciprocity follows the pattern, 'You helped someone, and I will help you'. The direction of cooperation is reversed in upstream reciprocity, which instead follows the pattern, 'You helped me, and I will help someone else'. In reality, these two types of indirect reciprocity often occur in combination. However, upstream and downstream reciprocity have mostly been studied theoretically in isolation. Here, we propose a new model that integrates both types. We apply the standard giving-game framework of indirect reciprocity and analyze the model by means of evolutionary game theory. We show that the model can result in the stable coexistence of altruistic reciprocators and free riders in well-mixed populations. We also found that considering inattention in the assessment rule can strengthen the stability of this mixed equilibrium, even resulting in a global attractor. Our results indicate that the cycles of forwarding help and rewarding help need to be established for creating and maintaining diversity and inclusion in a society. △ Less

Submitted 8 March, 2023; originally announced March 2023.

Comments: 14 pages, 4 figures, 2 tables

Journal ref: Games 2024, 15(2),15

The Borel transform of the WKB solution to the Pearcey system

Authors: Takashi Aoki, Takao Suzuki, Shofu Uchida

Abstract: We study the system of partial differential equations which characterizes the Pearcey integral from the viewpoint of the exact WKB analysis. It is shown that the Borel transform of the WKB solutions to the system can be written as a linear combination of the branches of an algebraic function of degree $4$. Moreover, some connection formulas for the system are given. We study the system of partial differential equations which characterizes the Pearcey integral from the viewpoint of the exact WKB analysis. It is shown that the Borel transform of the WKB solutions to the system can be written as a linear combination of the branches of an algebraic function of degree $4$. Moreover, some connection formulas for the system are given. △ Less

Submitted 5 March, 2023; originally announced March 2023.

Comments: 22 pages, 15 figures

MSC Class: 33C70; 34E20; 34M60

Cluster-Guided Semi-Supervised Domain Adaptation for Imbalanced Medical Image Classification

Authors: Shota Harada, Ryoma Bise, Kengo Araki, Akihiko Yoshizawa, Kazuhiro Terada, Mariyo Kurata, Naoki Nakajima, Hiroyuki Abe, Tetsuo Ushiku, Seiichi Uchida

Abstract: Semi-supervised domain adaptation is a technique to build a classifier for a target domain by modifying a classifier in another (source) domain using many unlabeled samples and a small number of labeled samples from the target domain. In this paper, we develop a semi-supervised domain adaptation method, which has robustness to class-imbalanced situations, which are common in medical image classifi… ▽ More Semi-supervised domain adaptation is a technique to build a classifier for a target domain by modifying a classifier in another (source) domain using many unlabeled samples and a small number of labeled samples from the target domain. In this paper, we develop a semi-supervised domain adaptation method, which has robustness to class-imbalanced situations, which are common in medical image classification tasks. For robustness, we propose a weakly-supervised clustering pipeline to obtain high-purity clusters and utilize the clusters in representation learning for domain adaptation. The proposed method showed state-of-the-art performance in the experiment using severely class-imbalanced pathological image patches. △ Less

Submitted 2 March, 2023; originally announced March 2023.

Discovery of Orbital Ordering in Bi2Sr2CaCu2O8+x

Authors: Shuqiu Wang, Niall Kennedy, Kazuhiro Fujita, Shin-ichi Uchida, Hiroshi Eisaki, Peter D. Johnson, J. C. Séamus Davis, Shane M. O'Mahony

Abstract: The primordial ingredient of cuprate superconductivity is the CuO2 unit cell. Here, theoretical attention usually concentrates on the intra-atom Coulombic interactions dominating the 3d^9 and 3d^10 configurations of each copper ion. However, if Coulombic interactions also occur between electrons of the 2p^6 orbitals of each planar oxygen atom, spontaneous orbital ordering may split their energy le… ▽ More The primordial ingredient of cuprate superconductivity is the CuO2 unit cell. Here, theoretical attention usually concentrates on the intra-atom Coulombic interactions dominating the 3d^9 and 3d^10 configurations of each copper ion. However, if Coulombic interactions also occur between electrons of the 2p^6 orbitals of each planar oxygen atom, spontaneous orbital ordering may split their energy levels. This long predicted intra-unit cell symmetry breaking should then generate an orbital ordered phase, for which the charge-transfer energy E separating the 2p^6 and 3d^10orbitals is distinct for the two oxygen atoms. Here we introduce sublattice resolved E(r) imaging techniques to CuO2 studies and discover intra-unit-cell rotational symmetry breaking of E(r), with energy-level splitting between the two oxygen atoms on the 50 meV scale. Spatially, this state is arranged in disordered Ising domains of orthogonally oriented orbital order that appear bounded by dopant ions, and within whose domain walls low energy electronic quadrupolar two-level systems occur. Overall, these data reveal a Q=0 orbitally ordered state that splits the energy levels of the oxygen orbitals by ~50 meV, in underdoped CuO2. △ Less

Submitted 11 November, 2023; v1 submitted 28 February, 2023; originally announced February 2023.

Comments: Manuscript: 21 pages, 4 figures. Methods: 40 pages, 23 figures

Journal ref: Nature Materials, 23, 492, 2024

Disease Severity Regression with Continuous Data Augmentation

Authors: Shumpei Takezaki, Kiyohito Tanaka, Seiichi Uchida, Takeaki Kadota

Abstract: Disease severity regression by a convolutional neural network (CNN) for medical images requires a sufficient number of image samples labeled with severity levels. Conditional generative adversarial network (cGAN)-based data augmentation (DA) is a possible solution, but it encounters two issues. The first issue is that existing cGANs cannot deal with real-valued severity levels as their conditions,… ▽ More Disease severity regression by a convolutional neural network (CNN) for medical images requires a sufficient number of image samples labeled with severity levels. Conditional generative adversarial network (cGAN)-based data augmentation (DA) is a possible solution, but it encounters two issues. The first issue is that existing cGANs cannot deal with real-valued severity levels as their conditions, and the second is that the severity of the generated images is not fully reliable. We propose continuous DA as a solution to the two issues. Our method uses continuous severity GAN to generate images at real-valued severity levels and dataset-disjoint multi-objective optimization to deal with the second issue. Our method was evaluated for estimating ulcerative colitis (UC) severity of endoscopic images and achieved higher classification performance than conventional DA methods. △ Less

Submitted 24 February, 2023; originally announced February 2023.

Comments: Accepted at ISBI2023

Learning from Label Proportion with Online Pseudo-Label Decision by Regret Minimization

Authors: Shinnosuke Matsuo, Ryoma Bise, Seiichi Uchida, Daiki Suehiro

Abstract: This paper proposes a novel and efficient method for Learning from Label Proportions (LLP), whose goal is to train a classifier only by using the class label proportions of instance sets, called bags. We propose a novel LLP method based on an online pseudo-labeling method with regret minimization. As opposed to the previous LLP methods, the proposed method effectively works even if the bag sizes a… ▽ More This paper proposes a novel and efficient method for Learning from Label Proportions (LLP), whose goal is to train a classifier only by using the class label proportions of instance sets, called bags. We propose a novel LLP method based on an online pseudo-labeling method with regret minimization. As opposed to the previous LLP methods, the proposed method effectively works even if the bag sizes are large. We demonstrate the effectiveness of the proposed method using some benchmark datasets. △ Less

Submitted 17 February, 2023; originally announced February 2023.

Comments: Accepted at ICASSP2023

Heat equation on the hypergraph containing vertices with given data

Authors: Takeshi Fukao, Masahiro Ikeda, Shun Uchida

Abstract: This paper is concerned with the Cauchy problem of a multivalued ordinary differential equation governed by the hypergraph Laplacian, which describes the diffusion of ``heat'' or ``particles'' on the vertices of hypergraph. We consider the case where the heat on several vertices are manipulated internally by the observer, namely, are fixed by some given functions. This situation can be reduced to… ▽ More This paper is concerned with the Cauchy problem of a multivalued ordinary differential equation governed by the hypergraph Laplacian, which describes the diffusion of ``heat'' or ``particles'' on the vertices of hypergraph. We consider the case where the heat on several vertices are manipulated internally by the observer, namely, are fixed by some given functions. This situation can be reduced to a nonlinear evolution equation associated with a time-dependent subdifferential operator, whose solvability has been investigated in numerous previous researches. In this paper, however, we give an alternative proof of the solvability in order to avoid some complicated calculations arising from the chain rule for the time-dependent subdifferential. As for results which cannot be assured by the known abstract theory, we also discuss the continuous dependence of solution on the given data and the time-global behavior of solution. △ Less

Submitted 11 December, 2022; originally announced December 2022.

Comments: 19 pages, 3 figures

MSC Class: 34G25; 05C65; 47J30; 47J35

CEFR-Based Sentence Difficulty Annotation and Assessment

Authors: Yuki Arase, Satoru Uchida, Tomoyuki Kajiwara

Abstract: Controllable text simplification is a crucial assistive technique for language learning and teaching. One of the primary factors hindering its advancement is the lack of a corpus annotated with sentence difficulty levels based on language ability descriptions. To address this problem, we created the CEFR-based Sentence Profile (CEFR-SP) corpus, containing 17k English sentences annotated with the l… ▽ More Controllable text simplification is a crucial assistive technique for language learning and teaching. One of the primary factors hindering its advancement is the lack of a corpus annotated with sentence difficulty levels based on language ability descriptions. To address this problem, we created the CEFR-based Sentence Profile (CEFR-SP) corpus, containing 17k English sentences annotated with the levels based on the Common European Framework of Reference for Languages assigned by English-education professionals. In addition, we propose a sentence-level assessment model to handle unbalanced level distribution because the most basic and highly proficient sentences are naturally scarce. In the experiments in this study, our method achieved a macro-F1 score of 84.5% in the level assessment, thus outperforming strong baselines employed in readability assessment. △ Less

Submitted 21 October, 2022; originally announced October 2022.

Comments: EMNLP 2022

Depth Contrast: Self-Supervised Pretraining on 3DPM Images for Mining Material Classification

Authors: Prakash Chandra Chhipa, Richa Upadhyay, Rajkumar Saini, Lars Lindqvist, Richard Nordenskjold, Seiichi Uchida, Marcus Liwicki

Abstract: This work presents a novel self-supervised representation learning method to learn efficient representations without labels on images from a 3DPM sensor (3-Dimensional Particle Measurement; estimates the particle size distribution of material) utilizing RGB images and depth maps of mining material on the conveyor belt. Human annotations for material categories on sensor-generated data are scarce a… ▽ More This work presents a novel self-supervised representation learning method to learn efficient representations without labels on images from a 3DPM sensor (3-Dimensional Particle Measurement; estimates the particle size distribution of material) utilizing RGB images and depth maps of mining material on the conveyor belt. Human annotations for material categories on sensor-generated data are scarce and cost-intensive. Currently, representation learning without human annotations remains unexplored for mining materials and does not leverage on utilization of sensor-generated data. The proposed method, Depth Contrast, enables self-supervised learning of representations without labels on the 3DPM dataset by exploiting depth maps and inductive transfer. The proposed method outperforms material classification over ImageNet transfer learning performance in fully supervised learning settings and achieves an F1 score of 0.73. Further, The proposed method yields an F1 score of 0.65 with an 11% improvement over ImageNet transfer learning performance in a semi-supervised setting when only 20% of labels are used in fine-tuning. Finally, the Proposed method showcases improved performance generalization on linear evaluation. The implementation of proposed method is available on GitHub. △ Less

Submitted 18 October, 2022; originally announced October 2022.

Comments: Accepted to CVF European Conference on Computer Vision Workshop(ECCVW 2022)

Deep Bayesian Active-Learning-to-Rank for Endoscopic Image Data

Authors: Takeaki Kadota, Hideaki Hayashi, Ryoma Bise, Kiyohito Tanaka, Seiichi Uchida

Abstract: Automatic image-based disease severity estimation generally uses discrete (i.e., quantized) severity labels. Annotating discrete labels is often difficult due to the images with ambiguous severity. An easier alternative is to use relative annotation, which compares the severity level between image pairs. By using a learning-to-rank framework with relative annotation, we can train a neural network… ▽ More Automatic image-based disease severity estimation generally uses discrete (i.e., quantized) severity labels. Annotating discrete labels is often difficult due to the images with ambiguous severity. An easier alternative is to use relative annotation, which compares the severity level between image pairs. By using a learning-to-rank framework with relative annotation, we can train a neural network that estimates rank scores that are relative to severity levels. However, the relative annotation for all possible pairs is prohibitive, and therefore, appropriate sample pair selection is mandatory. This paper proposes a deep Bayesian active-learning-to-rank, which trains a Bayesian convolutional neural network while automatically selecting appropriate pairs for relative annotation. We confirmed the efficiency of the proposed method through experiments on endoscopic images of ulcerative colitis. In addition, we confirmed that our method is useful even with the severe class imbalance because of its ability to select samples from minor classes automatically. △ Less

Submitted 5 August, 2022; originally announced August 2022.

Comments: 14 pages, 8 figures, accepted at MIUA 2022

Identification of a Nematic Pair Density Wave State in Bi2Sr2CaCu2O8+x

Authors: Weijiong Chen, Wangping Ren, Niall Kennedy, M. H. Hamidian, S. Uchida, H. Eisaki, Peter. D. Johnson, Shane O Mahony, J. C. Seamus Davis

Abstract: Electron-pair density wave (PDW) states are now an intense focus of research in the field of cuprate correlated superconductivity. PDWs exhibit periodically modulating superconductive electron pairing which can be visualized directly using scanned Josephson tunneling microscopy (SJTM). Although from theory, intertwining the d-wave superconducting (DSC) and PDW order parameters allows a plethora of… ▽ More Electron-pair density wave (PDW) states are now an intense focus of research in the field of cuprate correlated superconductivity. PDWs exhibit periodically modulating superconductive electron pairing which can be visualized directly using scanned Josephson tunneling microscopy (SJTM). Although from theory, intertwining the d-wave superconducting (DSC) and PDW order parameters allows a plethora of global electron-pair orders to appear, which one actually occurs in the various cuprates is unknown. Here we use SJTM to visualize the interplay of PDW and DSC states in Bi2Sr2CaCu2O8+x at a carrier density where the charge density wave (CDW) modulations are virtually nonexistent. Simultaneous visualization of their amplitudes reveals that the intertwined PDW and DSC are mutually attractive states. Then, by separately imaging the electron-pair density modulations of the two orthogonal PDWs, we discover a robust nematic PDW state. Its spatial arrangement entails Ising domains of opposite nematicity, each consisting primarily of unidirectional and lattice commensurate electron-pair density modulations. Further, we demonstrate by direct imaging that the scattering resonances identifying Zn impurity atom sites occur predominantly within boundaries between these domains. This implies that the nematic PDW state is pinned by Zn atoms, as was recently proposed (Lozano et al, PHYSICAL REVIEW B 103, L020502 (2021)). Taken in combination, these data indicate that the PDW in Bi2Sr2CaCu2O8+x is a vestigial nematic pair density wave state (J. Wardh and M. Granath arXiv:2203.08250v1). △ Less

Submitted 8 June, 2022; originally announced June 2022.

Comments: 15 pages, 4 figures

MontageGAN: Generation and Assembly of Multiple Components by GANs

Authors: Chean Fei Shee, Seiichi Uchida

Abstract: A multi-layer image is more valuable than a single-layer image from a graphic designer's perspective. However, most of the proposed image generation methods so far focus on single-layer images. In this paper, we propose MontageGAN, which is a Generative Adversarial Networks (GAN) framework for generating multi-layer images. Our method utilized a two-step approach consisting of local GANs and globa… ▽ More A multi-layer image is more valuable than a single-layer image from a graphic designer's perspective. However, most of the proposed image generation methods so far focus on single-layer images. In this paper, we propose MontageGAN, which is a Generative Adversarial Networks (GAN) framework for generating multi-layer images. Our method utilized a two-step approach consisting of local GANs and global GAN. Each local GAN learns to generate a specific image layer, and the global GAN learns the placement of each generated image layer. Through our experiments, we show the ability of our method to generate multi-layer images and estimate the placement of the generated image layers. △ Less

Submitted 31 May, 2022; originally announced May 2022.

Comments: Accepted at ICPR2022

An elementary proof of the Voros connection formula for WKB solutions to the Airy equation with a large parameter

Authors: Takashi Aoki, Takao Suzuki, Shofu Uchida

Abstract: The Voros connection formula for WKB solutions to the Airy equation with a large parameter is proved by using cubic equations. Some parts of the results are generalized to the Pearcey system, which is a two-variable version of the Airy equation, is given. The Voros connection formula for WKB solutions to the Airy equation with a large parameter is proved by using cubic equations. Some parts of the results are generalized to the Pearcey system, which is a two-variable version of the Airy equation, is given. △ Less

Submitted 5 May, 2022; originally announced May 2022.

Comments: 18 pages, 11 figures

MSC Class: 33C05; 34E05; 34E15; 34E20

Font Generation with Missing Impression Labels

Authors: Seiya Matsuda, Akisato Kimura, Seiichi Uchida

Abstract: Our goal is to generate fonts with specific impressions, by training a generative adversarial network with a font dataset with impression labels. The main difficulty is that font impression is ambiguous and the absence of an impression label does not always mean that the font does not have the impression. This paper proposes a font generation model that is robust against missing impression labels.… ▽ More Our goal is to generate fonts with specific impressions, by training a generative adversarial network with a font dataset with impression labels. The main difficulty is that font impression is ambiguous and the absence of an impression label does not always mean that the font does not have the impression. This paper proposes a font generation model that is robust against missing impression labels. The key ideas of the proposed method are (1)a co-occurrence-based missing label estimator and (2)an impression label space compressor. The first is to interpolate missing impression labels based on the co-occurrence of labels in the dataset and use them for training the model as completed label conditions. The second is an encoder-decoder module to compress the high-dimensional impression space into low-dimensional. We proved that the proposed model generates high-quality font images using multi-label data with missing labels through qualitative and quantitative evaluations. △ Less

Submitted 2 June, 2022; v1 submitted 19 March, 2022; originally announced March 2022.

Comments: Accepted ICPR2022

Revealing Reliable Signatures by Learning Top-Rank Pairs

Authors: Xiaotong Ji, Yan Zheng, Daiki Suehiro, Seiichi Uchida

Abstract: Signature verification, as a crucial practical documentation analysis task, has been continuously studied by researchers in machine learning and pattern recognition fields. In specific scenarios like confirming financial documents and legal instruments, ensuring the absolute reliability of signatures is of top priority. In this work, we proposed a new method to learn "top-rank pairs" for writer-in… ▽ More Signature verification, as a crucial practical documentation analysis task, has been continuously studied by researchers in machine learning and pattern recognition fields. In specific scenarios like confirming financial documents and legal instruments, ensuring the absolute reliability of signatures is of top priority. In this work, we proposed a new method to learn "top-rank pairs" for writer-independent offline signature verification tasks. By this scheme, it is possible to maximize the number of absolutely reliable signatures. More precisely, our method to learn top-rank pairs aims at pushing positive samples beyond negative samples, after pairing each of them with a genuine reference signature. In the experiment, BHSig-B and BHSig-H datasets are used for evaluation, on which the proposed model achieves overwhelming better pos@top (the ratio of absolute top positive samples to all of the positive samples) while showing encouraging performance on both Area Under the Curve (AUC) and accuracy. △ Less

Submitted 17 March, 2022; originally announced March 2022.

Optimal Rejection Function Meets Character Recognition Tasks

Authors: Xiaotong Ji, Yuchen Zheng, Daiki Suehiro, Seiichi Uchida

Abstract: In this paper, we propose an optimal rejection method for rejecting ambiguous samples by a rejection function. This rejection function is trained together with a classification function under the framework of Learning-with-Rejection (LwR). The highlights of LwR are: (1) the rejection strategy is not heuristic but has a strong background from a machine learning theory, and (2) the rejection functio… ▽ More In this paper, we propose an optimal rejection method for rejecting ambiguous samples by a rejection function. This rejection function is trained together with a classification function under the framework of Learning-with-Rejection (LwR). The highlights of LwR are: (1) the rejection strategy is not heuristic but has a strong background from a machine learning theory, and (2) the rejection function can be trained on an arbitrary feature space which is different from the feature space for classification. The latter suggests we can choose a feature space that is more suitable for rejection. Although the past research on LwR focused only on its theoretical aspect, we propose to utilize LwR for practical pattern classification tasks. Moreover, we propose to use features from different CNN layers for classification and rejection. Our extensive experiments of notMNIST classification and character/non-character classification demonstrate that the proposed method achieves better performance than traditional rejection strategies. △ Less

Submitted 17 March, 2022; originally announced March 2022.