Showing 1–25 of 25 results for author: Yoo, S J

Imperative vs. Declarative Programming Paradigms for Open-Universe Scene Generation

Authors: Maxim Gumin, Do Heon Han, Seung Jean Yoo, Aditya Ganeshan, R. Kenny Jones, Rio Aguina-Kang, Stewart Morris, Daniel Ritchie

Abstract: Synthesizing 3D scenes from open-vocabulary text descriptions is a challenging, important, and recently-popular application. One of its critical subproblems is layout generation: given a set of objects, lay them out to produce a scene matching the input description. Nearly all recent work adopts a declarative paradigm for this problem: using LLM to generate specification of constraints between obj… ▽ More Synthesizing 3D scenes from open-vocabulary text descriptions is a challenging, important, and recently-popular application. One of its critical subproblems is layout generation: given a set of objects, lay them out to produce a scene matching the input description. Nearly all recent work adopts a declarative paradigm for this problem: using LLM to generate specification of constraints between objects, then solving those constraints to produce the final layout. In contrast, we explore an alternative imperative paradigm, in which an LLM iteratively places objects, with each object's position and orientation computed as a function of previously-placed objects. The imperative approach allows for a simpler scene specification language while also handling a wider variety and larger complexity of scenes. We further improve the robustness of our imperative scheme by developing an error correction mechanism that iteratively improves the scene's validity while staying as close as possible the original layout generated by the LLM. In forced-choice perceptual studies, participants preferred layouts generated by our imperative approach 82% and 94% of the time, respectively, when compared against two declarative layout generation methods. We also present a simple, automated evaluation metric for 3D scene layout generation that aligns well with human preferences. △ Less

Submitted 7 April, 2025; originally announced April 2025.

Structure-from-Sherds++: Robust Incremental 3D Reassembly of Axially Symmetric Pots from Unordered and Mixed Fragment Collections

Authors: Seong Jong Yoo, Sisung Liu, Muhammad Zeeshan Arshad, Jinhyeok Kim, Young Min Kim, Yiannis Aloimonos, Cornelia Fermuller, Kyungdon Joo, Jinwook Kim, Je Hyeong Hong

Abstract: Reassembling multiple axially symmetric pots from fragmentary sherds is crucial for cultural heritage preservation, yet it poses significant challenges due to thin and sharp fracture surfaces that generate numerous false positive matches and hinder large-scale puzzle solving. Existing global approaches, which optimize all potential fragment pairs simultaneously or data-driven models, are prone to… ▽ More Reassembling multiple axially symmetric pots from fragmentary sherds is crucial for cultural heritage preservation, yet it poses significant challenges due to thin and sharp fracture surfaces that generate numerous false positive matches and hinder large-scale puzzle solving. Existing global approaches, which optimize all potential fragment pairs simultaneously or data-driven models, are prone to local minima and face scalability issues when multiple pots are intermixed. Motivated by Structure-from-Motion (SfM) for 3D reconstruction from multiple images, we propose an efficient reassembly method for axially symmetric pots based on iterative registration of one sherd at a time, called Structure-from-Sherds++ (SfS++). Our method extends beyond simple replication of incremental SfM and leverages multi-graph beam search to explore multiple registration paths. This allows us to effectively filter out indistinguishable false matches and simultaneously reconstruct multiple pots without requiring prior information such as base or the number of mixed objects. Our approach achieves 87% reassembly accuracy on a dataset of 142 real fragments from 10 different pots, outperforming other methods in handling complex fracture patterns with mixed datasets and achieving state-of-the-art performance. Code and results can be found in our project page https://sj-yoo.info/sfs/. △ Less

Submitted 18 February, 2025; originally announced February 2025.

Comments: 24 pages

Roadmap on Neuromorphic Photonics

Authors: Daniel Brunner, Bhavin J. Shastri, Mohammed A. Al Qadasi, H. Ballani, Sylvain Barbay, Stefano Biasi, Peter Bienstman, Simon Bilodeau, Wim Bogaerts, Fabian Böhm, G. Brennan, Sonia Buckley, Xinlun Cai, Marcello Calvanese Strinati, B. Canakci, Benoit Charbonnier, Mario Chemnitz, Yitong Chen, Stanley Cheung, Jeff Chiles, Suyeon Choi, Demetrios N. Christodoulides, Lukas Chrostowski, J. Chu, J. H. Clegg , et al. (125 additional authors not shown)

Abstract: This roadmap consolidates recent advances while exploring emerging applications, reflecting the remarkable diversity of hardware platforms, neuromorphic concepts, and implementation philosophies reported in the field. It emphasizes the critical role of cross-disciplinary collaboration in this rapidly evolving field. This roadmap consolidates recent advances while exploring emerging applications, reflecting the remarkable diversity of hardware platforms, neuromorphic concepts, and implementation philosophies reported in the field. It emphasizes the critical role of cross-disciplinary collaboration in this rapidly evolving field. △ Less

Submitted 16 January, 2025; v1 submitted 14 January, 2025; originally announced January 2025.

VioPose: Violin Performance 4D Pose Estimation by Hierarchical Audiovisual Inference

Authors: Seong Jong Yoo, Snehesh Shrestha, Irina Muresanu, Cornelia Fermüller

Abstract: Musicians delicately control their bodies to generate music. Sometimes, their motions are too subtle to be captured by the human eye. To analyze how they move to produce the music, we need to estimate precise 4D human pose (3D pose over time). However, current state-of-the-art (SoTA) visual pose estimation algorithms struggle to produce accurate monocular 4D poses because of occlusions, partial vi… ▽ More Musicians delicately control their bodies to generate music. Sometimes, their motions are too subtle to be captured by the human eye. To analyze how they move to produce the music, we need to estimate precise 4D human pose (3D pose over time). However, current state-of-the-art (SoTA) visual pose estimation algorithms struggle to produce accurate monocular 4D poses because of occlusions, partial views, and human-object interactions. They are limited by the viewing angle, pixel density, and sampling rate of the cameras and fail to estimate fast and subtle movements, such as in the musical effect of vibrato. We leverage the direct causal relationship between the music produced and the human motions creating them to address these challenges. We propose VioPose: a novel multimodal network that hierarchically estimates dynamics. High-level features are cascaded to low-level features and integrated into Bayesian updates. Our architecture is shown to produce accurate pose sequences, facilitating precise motion analysis, and outperforms SoTA. As part of this work, we collected the largest and the most diverse calibrated violin-playing dataset, including video, sound, and 3D motion capture poses. Code and dataset can be found in our project page \url{https://sj-yoo.info/viopose/}. △ Less

Submitted 25 November, 2024; v1 submitted 19 November, 2024; originally announced November 2024.

Comments: Accepted by WACV 2025 in Round 1. First two authors contributed equally

High Performance Three-Terminal Thyristor RAM with a P+/P/N/P/N/N+ Doping Profile on a Silicon-Photonic CMOS Platform

Authors: Changseob Lee, Ikhyeon Kwon, Anirban Samanta, Siwei Li, S. J. Ben Yoo

Abstract: 3T TRAM with doping profile (P+PNPNN+) is experimentally demonstrated on a silicon photonic platform. By using additional implant layers, this device provides excellent memory performance compared to the conventional structure (PNPN). TCAD is used to reflect the physical behavior, and the high-speed memory operations are described through the model. 3T TRAM with doping profile (P+PNPNN+) is experimentally demonstrated on a silicon photonic platform. By using additional implant layers, this device provides excellent memory performance compared to the conventional structure (PNPN). TCAD is used to reflect the physical behavior, and the high-speed memory operations are described through the model. △ Less

Submitted 11 September, 2024; originally announced September 2024.

Comments: 4 pages, 15 figures

TEGRA -- Scaling Up Terascale Graph Processing with Disaggregated Computing

Authors: William Shaddix, Mahyar Samani, Marjan Fariborz, S. J. Ben Yoo, Jason Lowe-Power, Venkatesh Akella

Abstract: Graphs are essential for representing relationships in various domains, driving modern AI applications such as graph analytics and neural networks across science, engineering, cybersecurity, transportation, and economics. However, the size of modern graphs are rapidly expanding, posing challenges for traditional CPUs and GPUs in meeting real-time processing demands. As a result, hardware accelerat… ▽ More Graphs are essential for representing relationships in various domains, driving modern AI applications such as graph analytics and neural networks across science, engineering, cybersecurity, transportation, and economics. However, the size of modern graphs are rapidly expanding, posing challenges for traditional CPUs and GPUs in meeting real-time processing demands. As a result, hardware accelerators for graph processing have been proposed. However, the largest graphs that can be handled by these systems is still modest often targeting Twitter graph(1.4B edges approximately). This paper aims to address this limitation by developing a graph accelerator capable of terascale graph processing. Scale out architectures, architectures where nodes are replicated to expand to larger datasets, are natural for handling larger graphs. We argue that this approach is not appropriate for very large-scale graphs because it leads to under utilization of both memory resources and compute resources. Additionally, vertex and edge processing have different access patterns. Communication overheads also pose further challenges in designing scalable architectures. To overcome these issues, this paper proposes TEGRA, a scale-up architecture for terascale graph processing. TEGRA leverages a composable computing system with disaggregated resources and a communication architecture inspired by Active Messages. By employing direct communication between cores and optimizing memory interconnect utilization, TEGRA effectively reduces communication overhead and improves resource utilization, therefore enabling efficient processing of terascale graphs. △ Less

Submitted 3 April, 2024; originally announced April 2024.

Comments: Presented at the 3rd Workshop on Heterogeneous Composable and Disaggregated Systems (HCDS 2024)

Towards Reverse-Engineering the Brain: Brain-Derived Neuromorphic Computing Approach with Photonic, Electronic, and Ionic Dynamicity in 3D integrated circuits

Authors: S. J. Ben Yoo, Luis El-Srouji, Suman Datta, Shimeng Yu, Jean Anne Incorvia, Alberto Salleo, Volker Sorger, Juejun Hu, Lionel C Kimerling, Kristofer Bouchard, Joy Geng, Rishidev Chaudhuri, Charan Ranganath, Randall O'Reilly

Abstract: The human brain has immense learning capabilities at extreme energy efficiencies and scale that no artificial system has been able to match. For decades, reverse engineering the brain has been one of the top priorities of science and technology research. Despite numerous efforts, conventional electronics-based methods have failed to match the scalability, energy efficiency, and self-supervised lea… ▽ More The human brain has immense learning capabilities at extreme energy efficiencies and scale that no artificial system has been able to match. For decades, reverse engineering the brain has been one of the top priorities of science and technology research. Despite numerous efforts, conventional electronics-based methods have failed to match the scalability, energy efficiency, and self-supervised learning capabilities of the human brain. On the other hand, very recent progress in the development of new generations of photonic and electronic memristive materials, device technologies, and 3D electronic-photonic integrated circuits (3D EPIC ) promise to realize new brain-derived neuromorphic systems with comparable connectivity, density, energy-efficiency, and scalability. When combined with bio-realistic learning algorithms and architectures, it may be possible to realize an 'artificial brain' prototype with general self-learning capabilities. This paper argues the possibility of reverse-engineering the brain through architecting a prototype of a brain-derived neuromorphic computing system consisting of artificial electronic, ionic, photonic materials, devices, and circuits with dynamicity resembling the bio-plausible molecular, neuro/synaptic, neuro-circuit, and multi-structural hierarchical macro-circuits of the brain based on well-tested computational models. We further argue the importance of bio-plausible local learning algorithms applicable to the neuromorphic computing system that capture the flexible and adaptive unsupervised and self-supervised learning mechanisms central to human intelligence. Most importantly, we emphasize that the unique capabilities in brain-derived neuromorphic computing prototype systems will enable us to understand links between specific neuronal and network-level properties with system-level functioning and behavior. △ Less

Submitted 28 March, 2024; originally announced March 2024.

Comments: 15 pages, 12 figures

Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases

Authors: Rio Aguina-Kang, Maxim Gumin, Do Heon Han, Stewart Morris, Seung Jean Yoo, Aditya Ganeshan, R. Kenny Jones, Qiuhong Anna Wei, Kailiang Fu, Daniel Ritchie

Abstract: We present a system for generating indoor scenes in response to text prompts. The prompts are not limited to a fixed vocabulary of scene descriptions, and the objects in generated scenes are not restricted to a fixed set of object categories -- we call this setting indoor scene generation. Unlike most prior work on indoor scene generation, our system does not require a large training dataset of ex… ▽ More We present a system for generating indoor scenes in response to text prompts. The prompts are not limited to a fixed vocabulary of scene descriptions, and the objects in generated scenes are not restricted to a fixed set of object categories -- we call this setting indoor scene generation. Unlike most prior work on indoor scene generation, our system does not require a large training dataset of existing 3D scenes. Instead, it leverages the world knowledge encoded in pre-trained large language models (LLMs) to synthesize programs in a domain-specific layout language that describe objects and spatial relations between them. Executing such a program produces a specification of a constraint satisfaction problem, which the system solves using a gradient-based optimization scheme to produce object positions and orientations. To produce object geometry, the system retrieves 3D meshes from a database. Unlike prior work which uses databases of category-annotated, mutually-aligned meshes, we develop a pipeline using vision-language models (VLMs) to retrieve meshes from massive databases of un-annotated, inconsistently-aligned meshes. Experimental evaluations show that our system outperforms generative models trained on 3D data for traditional, closed-universe scene generation tasks; it also outperforms a recent LLM-based layout generation method on open-universe scene generation. △ Less

Submitted 4 February, 2024; originally announced March 2024.

Comments: See ancillary files for link to supplemental material

Experimental Demonstration of Imperfection-Agnostic Local Learning Rules on Photonic Neural Networks with Mach-Zehnder Interferometric Meshes

Authors: Luis El Srouji, Mehmet Berkay On, Yun-Jhu Lee, Mahmoud Abdelghany, S. J. Ben Yoo

Abstract: Mach-Zehnder Interferometric meshes are attractive for low-loss photonic matrix multiplication but are challenging to program. Using least-squares optimization of directional derivatives, we experimentally demonstrate that desired matrix updates can be implemented agnostic to hardware imperfections. \c{opyright} 2024 The Author(s) Mach-Zehnder Interferometric meshes are attractive for low-loss photonic matrix multiplication but are challenging to program. Using least-squares optimization of directional derivatives, we experimentally demonstrate that desired matrix updates can be implemented agnostic to hardware imperfections. \c{opyright} 2024 The Author(s) △ Less

Submitted 7 January, 2024; originally announced January 2024.

0.08 fF, 0.72 nA dark current, 91% Quantum Efficiency, 38 Gb/s Nano-photodetector on a 45 nm CMOS Silicon-Photonic Platform

Authors: Mingye Fu, S. J. Ben Yoo

Abstract: We demonstrated a Germanium-on-Silicon photodetector utilizing an asymmetric-Fabry-Perot resonator with 0.08 fF capacitance. The measurements at 1315.5 nm show 0.72 nA (3.40 nA) dark current, 0.93 A/W (0.96 A/W) responsivity, 36 Gb/s (38 Gb/s) operation at -1V (-2V) bias. We demonstrated a Germanium-on-Silicon photodetector utilizing an asymmetric-Fabry-Perot resonator with 0.08 fF capacitance. The measurements at 1315.5 nm show 0.72 nA (3.40 nA) dark current, 0.93 A/W (0.96 A/W) responsivity, 36 Gb/s (38 Gb/s) operation at -1V (-2V) bias. △ Less

Submitted 7 January, 2024; originally announced January 2024.

An unconventional platform for two-dimensional Kagome flat bands on semiconductor surfaces

Authors: Jae Hyuck Lee, GwanWoo Kim, Inkyung Song, Yejin Kim, Yeonjae Lee, Sung Jong Yoo, Deok-Yong Cho, Jun-Won Rhim, Jongkeun Jung, Gunn Kim, Changyoung Kim

Abstract: In condensed matter physics, the Kagome lattice and its inherent flat bands have attracted considerable attention for their potential to host a variety of exotic physical phenomena. Despite extensive efforts to fabricate thin films of Kagome materials aimed at modulating the flat bands through electrostatic gating or strain manipulation, progress has been limited. Here, we report the observation o… ▽ More In condensed matter physics, the Kagome lattice and its inherent flat bands have attracted considerable attention for their potential to host a variety of exotic physical phenomena. Despite extensive efforts to fabricate thin films of Kagome materials aimed at modulating the flat bands through electrostatic gating or strain manipulation, progress has been limited. Here, we report the observation of a novel $d$-orbital hybridized Kagome-derived flat band in Ag/Si(111) $\sqrt{3}\times\sqrt{3}$ as revealed by angle-resolved photoemission spectroscopy. Our findings indicate that silver atoms on a silicon substrate form a Kagome-like structure, where a delicate balance in the hopping parameters of the in-plane $d$-orbitals leads to destructive interference, resulting in a flat band. These results not only introduce a new platform for Kagome physics but also illuminate the potential for integrating metal-semiconductor interfaces into Kagome-related research, thereby opening a new avenue for exploring ideal two-dimensional Kagome systems. △ Less

Submitted 30 December, 2023; originally announced January 2024.

Comments: 7 pages, 4 figures

Demonstration of Programmable Brain-Inspired Optoelectronic Neuron in Photonic Spiking Neural Network with Neural Heterogeneity

Authors: Yun-Jhu Lee, Mehmet Berkay On, Luis El Srouji, Li Zhang, Mahmoud Abdelghany, S. J. Ben Yoo

Abstract: Photonic Spiking Neural Networks (PSNN) composed of the co-integrated CMOS and photonic elements can offer low loss, low power, highly-parallel, and high-throughput computing for brain-inspired neuromorphic systems. In addition, heterogeneity of neuron dynamics can also bring greater diversity and expressivity to brain-inspired networks, potentially allowing for the implementation of complex funct… ▽ More Photonic Spiking Neural Networks (PSNN) composed of the co-integrated CMOS and photonic elements can offer low loss, low power, highly-parallel, and high-throughput computing for brain-inspired neuromorphic systems. In addition, heterogeneity of neuron dynamics can also bring greater diversity and expressivity to brain-inspired networks, potentially allowing for the implementation of complex functions with fewer neurons. In this paper, we design, fabricate, and experimentally demonstrate an optoelectronic spiking neuron that can simultaneously achieve high programmability for heterogeneous biological neural networks and maintain high-speed computing. We demonstrate that our neuron can be programmed to tune four essential parameters of neuron dynamics under 1GSpike/s input spiking pattern signals. A single neuron circuit can be tuned to output three spiking patterns, including chattering behaviors. The PSNN consisting of the optoelectronic spiking neuron and a Mach-Zehnder interferometer (MZI) mesh synaptic network achieves 89.3% accuracy on the Iris dataset. Our neuron power consumption is 1.18 pJ/spike output, mainly limited by the power efficiency of the vertical-cavity-lasers, optical coupling efficiency, and the 45 nm CMOS platform used in this experiment, and is predicted to achieve 36.84 fJ/spike output with a 7 nm CMOS platform (e.g. ASAP7) integrated with silicon photonics containing on-chip micron-scale lasers. △ Less

Submitted 26 November, 2023; originally announced November 2023.

AcTExplore: Active Tactile Exploration of Unknown Objects

Authors: Amir-Hossein Shahidzadeh, Seong Jong Yoo, Pavan Mantripragada, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos

Abstract: Tactile exploration plays a crucial role in understanding object structures for fundamental robotics tasks such as grasping and manipulation. However, efficiently exploring such objects using tactile sensors is challenging, primarily due to the large-scale unknown environments and limited sensing coverage of these sensors. To this end, we present AcTExplore, an active tactile exploration method dr… ▽ More Tactile exploration plays a crucial role in understanding object structures for fundamental robotics tasks such as grasping and manipulation. However, efficiently exploring such objects using tactile sensors is challenging, primarily due to the large-scale unknown environments and limited sensing coverage of these sensors. To this end, we present AcTExplore, an active tactile exploration method driven by reinforcement learning for object reconstruction at scales that automatically explores the object surfaces in a limited number of steps. Through sufficient exploration, our algorithm incrementally collects tactile data and reconstructs 3D shapes of the objects as well, which can serve as a representation for higher-level downstream tasks. Our method achieves an average of 95.97% IoU coverage on unseen YCB objects while just being trained on primitive shapes. Project Webpage: https://prg.cs.umd.edu/AcTExplore △ Less

Submitted 20 June, 2024; v1 submitted 12 October, 2023; originally announced October 2023.

Comments: 8 pages, 6 figures, Accepted to ICRA 2024

Programmable Integrated Photonics for Topological Hamiltonians

Authors: Mehmet Berkay On, Farshid Ashtiani, David Sanchez-Jacome, Daniel Perez-Lopez, S. J. Ben Yoo, Andrea Blanco-Redondo

Abstract: A variety of topological Hamiltonians have been demonstrated in photonic platforms, leading to fundamental discoveries and enhanced robustness in applications such as lasing, sensing, and quantum technologies. To date, each topological photonic platform implements a specific type of Hamiltonian with inexistent or limited reconfigurability. Here, we propose and demonstrate different topological mod… ▽ More A variety of topological Hamiltonians have been demonstrated in photonic platforms, leading to fundamental discoveries and enhanced robustness in applications such as lasing, sensing, and quantum technologies. To date, each topological photonic platform implements a specific type of Hamiltonian with inexistent or limited reconfigurability. Here, we propose and demonstrate different topological models by using the same reprogrammable integrated photonics platform, consisting of a hexagonal mesh of silicon Mach-Zehnder interferometers with phase-shifters. We specifically demonstrate a one-dimensional Su-Schrieffer-Heeger Hamiltonian supporting a localized topological edge mode and a higher-order topological insulator based on a two-dimensional breathing Kagome Hamiltonian with three corner states. These results highlight a nearly universal platform for topological models that may fast-track research progress toward applications of topological photonics and other coupled systems. △ Less

Submitted 10 July, 2023; originally announced July 2023.

Quantum Wrapper Networking

Authors: S. J. Ben Yoo, Sandeep Kumar Singh, Mehmet Berkay On, Gamze Gul, Gregory S. Kanter, Roberto Proietti, Prem Kumar

Abstract: We introduce a new concept of Quantum Wrapper Networking, which enables control, management, and operation of quantum networks that can co-exist with classical networks while keeping the requirements for quantum networks intact. The quantum wrapper networks (QWNs) enable the transparent and interoperable transportation of quantum wrapper datagrams consisting of quantum payloads and, notably, class… ▽ More We introduce a new concept of Quantum Wrapper Networking, which enables control, management, and operation of quantum networks that can co-exist with classical networks while keeping the requirements for quantum networks intact. The quantum wrapper networks (QWNs) enable the transparent and interoperable transportation of quantum wrapper datagrams consisting of quantum payloads and, notably, classical headers to facilitate the datagram switching without measuring or disturbing the qubits of the quantum payload. Furthermore, QWNs can utilize the common network control and management for performance monitoring on the classical header and infer the quantum channel quality. △ Less

Submitted 30 April, 2023; originally announced May 2023.

Comments: This manuscript has 5 figures and 7 pages. The very early version of the initial concept of Quantum Wrapper Networking was published in IEEE Photonics Conference 2021 followed by Optical Communications Conference 2023 Paper Tu3H.4 (referenced as [12])

New Trends in Photonic Switching and Optical Network Architecture for Data Centre and Computing Systems

Authors: S. J. Ben Yoo

Abstract: AI/ML for data centres and data centres for AI/ML are defining new trends in cloud computing. Disaggregated heterogeneous reconfigurable computing systems realized by photonic interconnects and photonic switching expect greatly enhanced throughput and energy-efficiency for AI/ML workloads, especially when aided by an AI/ML control plane. AI/ML for data centres and data centres for AI/ML are defining new trends in cloud computing. Disaggregated heterogeneous reconfigurable computing systems realized by photonic interconnects and photonic switching expect greatly enhanced throughput and energy-efficiency for AI/ML workloads, especially when aided by an AI/ML control plane. △ Less

Submitted 19 September, 2022; originally announced September 2022.

Scalable Nanophotonic-Electronic Spiking Neural Networks

Authors: Luis El Srouji, Yun-Jhu Lee, Mehmet Berkay On, Li Zhang, S. J. Ben Yoo

Abstract: Spiking neural networks (SNN) provide a new computational paradigm capable of highly parallelized, real-time processing. Photonic devices are ideal for the design of high-bandwidth, parallel architectures matching the SNN computational paradigm. Co-integration of CMOS and photonic elements allow low-loss photonic devices to be combined with analog electronics for greater flexibility of nonlinear c… ▽ More Spiking neural networks (SNN) provide a new computational paradigm capable of highly parallelized, real-time processing. Photonic devices are ideal for the design of high-bandwidth, parallel architectures matching the SNN computational paradigm. Co-integration of CMOS and photonic elements allow low-loss photonic devices to be combined with analog electronics for greater flexibility of nonlinear computational elements. As such, we designed and simulated an optoelectronic spiking neuron circuit on a monolithic silicon photonics (SiPh) process that replicates useful spiking behaviors beyond the leaky integrate-and-fire (LIF). Additionally, we explored two learning algorithms with the potential for on-chip learning using Mach-Zehnder Interferometric (MZI) meshes as synaptic interconnects. A variation of Random Backpropagation (RPB) was experimentally demonstrated on-chip and matched the performance of a standard linear regression on a simple classification task. Meanwhile, the Contrastive Hebbian Learning (CHL) rule was applied to a simulated neural network composed of MZI meshes for a random input-output mapping task. The CHL-trained MZI network performed better than random guessing but does not match the performance of the ideal neural network (without the constraints imposed by the MZI meshes). Through these efforts, we demonstrate that co-integrated CMOS and SiPh technologies are well-suited to the design of scalable SNN computing architectures. △ Less

Submitted 28 August, 2022; originally announced August 2022.

Detection of Weyl Fermions and the Metal to Weyl-Semimetal phase transition in WTe$_2$ via broadband High Resolution NMR

Authors: Wassilios Papawassiliou, José P. Carvalho, Hae Jin Kim, Chang-Yeon Kim, Seung Jo Yoo, Jin Bae Lee, Saeed Alhassan, Savvas Orfanidis, Vassilios Psycharis, Marina Karagianni, Michael Fardis, Nikolaos Panopoulos, Georgios Papavassiliou, Andrew J. Pell

Abstract: Weyl Fermions (WFs) in the type-II Weyl Semimetal (WSM) WTe$_2$ are difficult to resolve experimentally because the Weyl bands disperse in an extremely narrow region of the (E-k) space. Here, by using DFT-assisted high-resolution $^{125}$Te solid-state NMR (ssNMR) in the temperature range $50$K - $700$K, we succeeded in detecting low energy WF excitations and monitor their evolution with temperatu… ▽ More Weyl Fermions (WFs) in the type-II Weyl Semimetal (WSM) WTe$_2$ are difficult to resolve experimentally because the Weyl bands disperse in an extremely narrow region of the (E-k) space. Here, by using DFT-assisted high-resolution $^{125}$Te solid-state NMR (ssNMR) in the temperature range $50$K - $700$K, we succeeded in detecting low energy WF excitations and monitor their evolution with temperature. Remarkably, WFs appear to emerge at T$\sim 120$K; at lower temperatures WTe$_2$ behaves as a metal. This intriguing metal-to-WSM phase transition is shown to be induced by the rapid raise of the Fermi level with temperature, crossing solely the electron and hole pockets in the low-T metallic phase, while crossing the Weyl bands near the nodal points - a prerequisite for the emergence of WFs - only for T$>120$K. △ Less

Submitted 6 December, 2021; v1 submitted 4 October, 2021; originally announced October 2021.

Comments: 23 pages (5 main article, 18 supplementary material), 16 figures (4 main article, 12 supplementary material), 1 table (supplementary material)

Izhikevich-Inspired Optoelectronic Neurons with Excitatory and Inhibitory Inputs for Energy-Efficient Photonic Spiking Neural Networks

Authors: Yun-jhu Lee, Mehmet Berkay On, Xian Xiao, Roberto Proietti, S. J. Ben Yoo

Abstract: We designed, prototyped, and experimentally demonstrated, for the first time to our knowledge, an optoelectronic spiking neuron inspired by the Izhikevich model incorporating both excitatory and inhibitory optical spiking inputs and producing optical spiking outputs accordingly. The optoelectronic neurons consist of three transistors acting as electrical spiking circuits, a vertical-cavity surface… ▽ More We designed, prototyped, and experimentally demonstrated, for the first time to our knowledge, an optoelectronic spiking neuron inspired by the Izhikevich model incorporating both excitatory and inhibitory optical spiking inputs and producing optical spiking outputs accordingly. The optoelectronic neurons consist of three transistors acting as electrical spiking circuits, a vertical-cavity surface-emitting laser (VCSEL) for optical spiking outputs, and two photodetectors for excitatory and inhibitory optical spiking inputs. Additional inclusion of capacitors and resistors complete the Izhikevich-inspired optoelectronic neurons, which receive excitatory and inhibitory optical spikes as inputs from other optoelectronic neurons. We developed a detailed optoelectronic neuron model in Verilog-A and simulated the circuit-level operation of various cases with excitatory input and inhibitory input signals. The experimental results closely resemble the simulated results and demonstrate how the excitatory inputs trigger the optical spiking outputs while the inhibitory inputs suppress the outputs. Utilizing the simulated neuron model, we conducted simulations using fully connected (FC) and convolutional neural networks (CNN). The simulation results using MNIST handwritten digits recognition show 90% accuracy on unsupervised learning and 97% accuracy on a supervised modified FC neural network. We further designed a nanoscale optoelectronic neuron utilizing quantum impedance conversion where a 200 aJ/spike input can trigger the output from on-chip nanolasers with 10 fJ/spike. The nanoscale neuron can support a fanout of ~80 or overcome 19 dB excess optical loss while running at 10 GSpikes/second in the neural network, which corresponds to 100x throughput and 1000x energy-efficiency improvement compared to state-of-art electrical neuromorphic hardware such as Loihi and NeuroGrid. △ Less

Submitted 2 May, 2021; originally announced May 2021.

Comments: 24 pages, 13 figures

DeepRMSA: A Deep Reinforcement Learning Framework for Routing, Modulation and Spectrum Assignment in Elastic Optical Networks

Authors: Xiaoliang Chen, Baojia Li, Roberto Proietti, Hongbo Lu, Zuqing Zhu, S. J. Ben Yoo

Abstract: This paper proposes DeepRMSA, a deep reinforcement learning framework for routing, modulation and spectrum assignment (RMSA) in elastic optical networks (EONs). DeepRMSA learns the correct online RMSA policies by parameterizing the policies with deep neural networks (DNNs) that can sense complex EON states. The DNNs are trained with experiences of dynamic lightpath provisioning. We first modify th… ▽ More This paper proposes DeepRMSA, a deep reinforcement learning framework for routing, modulation and spectrum assignment (RMSA) in elastic optical networks (EONs). DeepRMSA learns the correct online RMSA policies by parameterizing the policies with deep neural networks (DNNs) that can sense complex EON states. The DNNs are trained with experiences of dynamic lightpath provisioning. We first modify the asynchronous advantage actor-critic algorithm and present an episode-based training mechanism for DeepRMSA, namely, DeepRMSA-EP. DeepRMSA-EP divides the dynamic provisioning process into multiple episodes (each containing the servicing of a fixed number of lightpath requests) and performs training by the end of each episode. The optimization target of DeepRMSA-EP at each step of servicing a request is to maximize the cumulative reward within the rest of the episode. Thus, we obviate the need for estimating the rewards related to unknown future states. To overcome the instability issue in the training of DeepRMSA-EP due to the oscillations of cumulative rewards, we further propose a window-based flexible training mechanism, i.e., DeepRMSA-FLX. DeepRMSA-FLX attempts to smooth out the oscillations by defining the optimization scope at each step as a sliding window, and ensuring that the cumulative rewards always include rewards from a fixed number of requests. Evaluations with the two sample topologies show that DeepRMSA-FLX can effectively stabilize the training while achieving blocking probability reductions of more than 20.3% and 14.3%, when compared with the baselines. △ Less

Submitted 15 May, 2019; v1 submitted 6 May, 2019; originally announced May 2019.

Multimodal Deep Learning for Finance: Integrating and Forecasting International Stock Markets

Authors: Sang Il Lee, Seong Joon Yoo

Abstract: In today's increasingly international economy, return and volatility spillover effects across international equity markets are major macroeconomic drivers of stock dynamics. Thus, information regarding foreign markets is one of the most important factors in forecasting domestic stock prices. However, the cross-correlation between domestic and foreign markets is highly complex. Hence, it is extreme… ▽ More In today's increasingly international economy, return and volatility spillover effects across international equity markets are major macroeconomic drivers of stock dynamics. Thus, information regarding foreign markets is one of the most important factors in forecasting domestic stock prices. However, the cross-correlation between domestic and foreign markets is highly complex. Hence, it is extremely difficult to explicitly express this cross-correlation with a dynamical equation. In this study, we develop stock return prediction models that can jointly consider international markets, using multimodal deep learning. Our contributions are three-fold: (1) we visualize the transfer information between South Korea and US stock markets by using scatter plots; (2) we incorporate the information into the stock prediction models with the help of multimodal deep learning; (3) we conclusively demonstrate that the early and intermediate fusion models achieve a significant performance boost in comparison with the late fusion and single modality models. Our study indicates that jointly considering international stock markets can improve the prediction accuracy and deep neural networks are highly effective for such tasks. △ Less

Submitted 19 September, 2019; v1 submitted 15 March, 2019; originally announced March 2019.

A catalog of merging dwarf galaxies in the local universe

Authors: Sanjaya Paudel, Rory Smith, Suk Jin Yoo, Paula Calderón-Castillo, Pierre-Alain Duc

Abstract: We present the largest publicly available catalog of interacting dwarf galaxies. It includes 177 nearby merging dwarf galaxies of stellar mass M$_{*}$ $<$ 10$^{10}$M$_{\sun}$ and redshifts z $<$ 0.02. These galaxies are selected by visual inspection of publicly available archival imaging from two wide-field optical surveys (SDSS III and the Legacy Survey), and they possess low surface brightness f… ▽ More We present the largest publicly available catalog of interacting dwarf galaxies. It includes 177 nearby merging dwarf galaxies of stellar mass M$_{*}$ $<$ 10$^{10}$M$_{\sun}$ and redshifts z $<$ 0.02. These galaxies are selected by visual inspection of publicly available archival imaging from two wide-field optical surveys (SDSS III and the Legacy Survey), and they possess low surface brightness features that are likely the result of an interaction between dwarf galaxies. We list UV and optical photometric data which we use to estimate stellar masses and star formation rates. So far, the study of interacting dwarf galaxies has largely been done on an individual basis, and lacks a sufficiently large catalog to give statistics on the properties of interacting dwarf galaxies, and their role in the evolution of low mass galaxies. We expect that this public catalog can be used as a reference sample to investigate the effects of the tidal interaction on the evolution of star-formation, morphology/structure of dwarf galaxies. Our sample is overwhelmingly dominated by star-forming galaxies, and they are generally found significantly below the red-sequence in the color-magnitude relation. The number of early-type galaxies is only 3 out of 177. We classify them, according to observed low surface brightness features, into various categories including shells, stellar streams, loops, antennae or simply interacting. We find that dwarf-dwarf interactions tend to prefer the low density environment. Only 41 out of the 177 candidate dwarf-dwarf interaction systems have giant neighbors within a sky projected distance of 700 kpc and a line of sight radial velocity range $\pm$700 km/s and, compared to the LMC-SMC, they are generally located at much larger sky-projected distances from their nearest giant neighbor. △ Less

Submitted 18 July, 2018; originally announced July 2018.

Comments: 26 Pages, Accepted for publication in ApJS

Threshold-Based Portfolio: The Role of the Threshold and Its Applications

Authors: Sang Il Lee, Seong Joon Yoo

Abstract: This paper aims at developing a new method by which to build a data-driven portfolio featuring a target risk-return. We first present a comparative study of recurrent neural network models (RNNs), including a simple RNN, long short-term memory (LSTM), and gated recurrent unit (GRU) for selecting the best predictor to use in portfolio construction. The models are applied to the investment universe… ▽ More This paper aims at developing a new method by which to build a data-driven portfolio featuring a target risk-return. We first present a comparative study of recurrent neural network models (RNNs), including a simple RNN, long short-term memory (LSTM), and gated recurrent unit (GRU) for selecting the best predictor to use in portfolio construction. The models are applied to the investment universe consisted of ten stocks in the S&P500. The experimental results shows that LSTM outperforms the others in terms of hit ratio of one-month-ahead forecasts. We then build predictive threshold-based portfolios (TBPs) that are subsets of the universe satisfying given threshold criteria for the predicted returns. The TBPs are rebalanced monthly to restore equal weights to each security within the TBPs. We find that the risk and return profile of the realized TBP represents a monotonically increasing frontier on the risk-return plane, where the equally weighted portfolio (EWP) of all ten stocks plays a role in their lower bound. This shows the availability of TBPs in targeting specific risk-return levels, and an EWP based on all the assets plays a role in the reference portfolio of TBPs. In the process, thresholds play dominant roles in characterizing risk, return, and the prediction accuracy of the subset. The TBP is more data-driven in designing portfolio target risk and return than existing ones, in the sense that it requires no prior knowledge of finance such as financial assumptions, financial mathematics, or expert insights. In a practical application, we present the TBP management procedure for a time horizon extending over multiple time periods; we also discuss their application to mean-variance portfolios to reduce estimation risk. △ Less

Submitted 2 August, 2018; v1 submitted 28 September, 2017; originally announced September 2017.

Comments: 20 pages, 7 figures

NH3 adsorption on PtM (Fe, Co, Ni) surfaces: cooperating effects of charge transfer, magnetic ordering and lattice strain

Authors: Satadeep Bhattacharjee, S. J. Yoo, Umesh V. Waghmare, S. C. Lee

Abstract: Adsorption of a molecule or group with an atom which is less electronegative than oxygen (O) and directly interacting with the surface is very relevant to development of PtM (M=3d-transition metal) catalysts with high activity. Here, we present theoretical analysis of the adsorption of NH3 molecule (N being less electronegative than O) on (111) surfaces of PtM(Fe,Co,Ni) alloys using the first prin… ▽ More Adsorption of a molecule or group with an atom which is less electronegative than oxygen (O) and directly interacting with the surface is very relevant to development of PtM (M=3d-transition metal) catalysts with high activity. Here, we present theoretical analysis of the adsorption of NH3 molecule (N being less electronegative than O) on (111) surfaces of PtM(Fe,Co,Ni) alloys using the first principles density functional approach. We find that, while NH3-Pt interaction is stronger than that of NH3 with the elemental M-surfaces, it is weaker than the strength of interaction of NH3 with M-site on the surface of PtM alloy. △ Less

Submitted 14 January, 2016; originally announced January 2016.

Comments: Accepted in Chem. Phys. Lett

Epitaxial Growth of a Single-Crystal Hybridized Boron Nitride and Graphene layer on a Wide-Band Gap Semiconductor

Authors: Ha-Chul Shin, Yamujin Jang, Tae-Hoon Kim, Jun-Hae Lee, Dong-Hwa Oh, Sung Joon Ahn, Jae Hyun Lee, Youngkwon Moon, Ji-Hoon Park, Sung Jong Yoo, Chong-Yun Park, Dongmok Whang, Cheol-Woong Yang, Joung Real Ahn

Abstract: Vertical and lateral heterogeneous structures of two-dimensional (2D) materials have paved the way for pioneering studies on the physics and applications of 2D materials. A hybridized hexagonal boron nitride (h-BN) and graphene lateral structure, a heterogeneous 2D structure, has been fabricated on single-crystal metals or metal foils by chemical vapor deposition (CVD). However, once fabricated on… ▽ More Vertical and lateral heterogeneous structures of two-dimensional (2D) materials have paved the way for pioneering studies on the physics and applications of 2D materials. A hybridized hexagonal boron nitride (h-BN) and graphene lateral structure, a heterogeneous 2D structure, has been fabricated on single-crystal metals or metal foils by chemical vapor deposition (CVD). However, once fabricated on metals, the h-BN/graphene lateral structures require an additional transfer process for device applications, as reported for CVD graphene grown on metal foils. Here, we demonstrate that a single-crystal h-BN/graphene lateral structure can be epitaxially grown on a wide-gap semiconductor, SiC(0001). First, a single-crystal h-BN layer with the same orientation as bulk SiC was grown on a Si-terminated SiC substrate at 850 oC using borazine molecules. Second, when heated above 1150 oC in vacuum, the h-BN layer was partially removed and, subsequently, replaced with graphene domains. Interestingly, these graphene domains possess the same orientation as the h-BN layer, resulting in a single-crystal h-BN/graphene lateral structure on a whole sample area. For temperatures above 1600 oC, the single-crystal h-BN layer was completely replaced by the single-crystal graphene layer. The crystalline structure, electronic band structure, and atomic structure of the h-BN/graphene lateral structure were studied by using low energy electron diffraction, angle-resolved photoemission spectroscopy, and scanning tunneling microscopy, respectively. The h-BN/graphene lateral structure fabricated on a wide-gap semiconductor substrate can be directly applied to devices without a further transfer process, as reported for epitaxial graphene on a SiC substrate. △ Less

Submitted 12 June, 2015; originally announced June 2015.

Comments: 23 pages, 7 figures

Journal ref: Journal of the American Chemical Society, 137, 6897, 2015