-
An All-Optical General-Purpose CPU and Optical Computer Architecture
Authors:
Michael Kissner,
Leonardo Del Bino,
Felix Päsler,
Peter Caruana,
George Ghalanos
Abstract:
Energy efficiency of electronic digital processors is primarily limited by the energy consumption of electronic communication and interconnects. The industry is almost unanimously pushing towards replacing both long-haul, as well as local chip interconnects, using optics to drastically increase efficiency. In this paper, we explore what comes after the successful migration to optical interconnects…
▽ More
Energy efficiency of electronic digital processors is primarily limited by the energy consumption of electronic communication and interconnects. The industry is almost unanimously pushing towards replacing both long-haul, as well as local chip interconnects, using optics to drastically increase efficiency. In this paper, we explore what comes after the successful migration to optical interconnects, as with this inefficiency solved, the main source of energy consumption will be electronic digital computing, memory and electro-optical conversion. Our approach attempts to address all these issues by introducing efficient all-optical digital computing and memory, which in turn eliminates the need for electro-optical conversions. Here, we demonstrate for the first time a scheme to enable general purpose digital data processing in an integrated form and present our photonic integrated circuit (PIC) implementation. For this demonstration we implemented a URISC architecture capable of running any classical piece of software all-optically and present a comprehensive architectural framework for all-optical computing to go beyond.
△ Less
Submitted 23 September, 2024; v1 submitted 29 February, 2024;
originally announced March 2024.
-
A Neural-Symbolic Framework for Mental Simulation
Authors:
Michael Kissner
Abstract:
We present a neural-symbolic framework for observing the environment and continuously learning visual semantics and intuitive physics to reproduce them in an interactive simulation. The framework consists of five parts, a neural-symbolic hybrid network based on capsules for inverse graphics, an episodic memory to store observations, an interaction network for intuitive physics, a meta-learning age…
▽ More
We present a neural-symbolic framework for observing the environment and continuously learning visual semantics and intuitive physics to reproduce them in an interactive simulation. The framework consists of five parts, a neural-symbolic hybrid network based on capsules for inverse graphics, an episodic memory to store observations, an interaction network for intuitive physics, a meta-learning agent that continuously improves the framework and a querying language that acts as the framework's interface for simulation. By means of lifelong meta-learning, the capsule network is expanded and trained continuously, in order to better adapt to its environment with each iteration. This enables it to learn new semantics using a few-shot approach and with minimal input from an oracle over its lifetime. From what it learned through observation, the part for intuitive physics infers all the required physical properties of the objects in a scene, enabling predictions. Finally, a custom query language ties all parts together, which allows to perform various mental simulation tasks, such as navigation, sorting and simulation of a game environment, with which we illustrate the potential of our novel approach.
△ Less
Submitted 5 August, 2020;
originally announced August 2020.
-
Hacking Neural Networks: A Short Introduction
Authors:
Michael Kissner
Abstract:
A large chunk of research on the security issues of neural networks is focused on adversarial attacks. However, there exists a vast sea of simpler attacks one can perform both against and with neural networks. In this article, we give a quick introduction on how deep learning in security works and explore the basic methods of exploitation, but also look at the offensive capabilities deep learning…
▽ More
A large chunk of research on the security issues of neural networks is focused on adversarial attacks. However, there exists a vast sea of simpler attacks one can perform both against and with neural networks. In this article, we give a quick introduction on how deep learning in security works and explore the basic methods of exploitation, but also look at the offensive capabilities deep learning enabled tools provide. All presented attacks, such as backdooring, GPU-based buffer overflows or automated bug hunting, are accompanied by short open-source exercises for anyone to try out.
△ Less
Submitted 1 December, 2019; v1 submitted 18 November, 2019;
originally announced November 2019.
-
Adding Intuitive Physics to Neural-Symbolic Capsules Using Interaction Networks
Authors:
Michael Kissner,
Helmut Mayer
Abstract:
Many current methods to learn intuitive physics are based on interaction networks and similar approaches. However, they rely on information that has proven difficult to estimate directly from image data in the past. We aim to narrow this gap by inferring all the semantic information needed from raw pixel data in the form of a scene-graph. Our approach is based on neural-symbolic capsules, which id…
▽ More
Many current methods to learn intuitive physics are based on interaction networks and similar approaches. However, they rely on information that has proven difficult to estimate directly from image data in the past. We aim to narrow this gap by inferring all the semantic information needed from raw pixel data in the form of a scene-graph. Our approach is based on neural-symbolic capsules, which identify which objects in the scene are static, dynamic, elastic or rigid, possible joints between them, as well as their collision information. By integrating all this with interaction networks, we demonstrate how our method is able to learn intuitive physics directly from image sequences and apply its knowledge to new scenes and objects, resulting in an inverse-simulation pipeline.
△ Less
Submitted 22 June, 2019; v1 submitted 23 May, 2019;
originally announced May 2019.
-
A Neural-Symbolic Architecture for Inverse Graphics Improved by Lifelong Meta-Learning
Authors:
Michael Kissner,
Helmut Mayer
Abstract:
We follow the idea of formulating vision as inverse graphics and propose a new type of element for this task, a neural-symbolic capsule. It is capable of de-rendering a scene into semantic information feed-forward, as well as rendering it feed-backward. An initial set of capsules for graphical primitives is obtained from a generative grammar and connected into a full capsule network. Lifelong meta…
▽ More
We follow the idea of formulating vision as inverse graphics and propose a new type of element for this task, a neural-symbolic capsule. It is capable of de-rendering a scene into semantic information feed-forward, as well as rendering it feed-backward. An initial set of capsules for graphical primitives is obtained from a generative grammar and connected into a full capsule network. Lifelong meta-learning continuously improves this network's detection capabilities by adding capsules for new and more complex objects it detects in a scene using few-shot learning. Preliminary results demonstrate the potential of our novel approach.
△ Less
Submitted 4 September, 2019; v1 submitted 21 May, 2019;
originally announced May 2019.
