-
On-Chip Optical Convolutional Neural Networks
Authors:
Hengameh Bagherian,
Scott Skirlo,
Yichen Shen,
Huaiyu Meng,
Vladimir Ceperic,
Marin Soljacic
Abstract:
Convolutional Neural Networks (CNNs) are a class of Artificial Neural Networks(ANNs) that employ the method of convolving input images with filter-kernels for object recognition and classification purposes. In this paper, we propose a photonics circuit architecture which could consume a fraction of energy per inference compared with state of the art electronics.
Convolutional Neural Networks (CNNs) are a class of Artificial Neural Networks(ANNs) that employ the method of convolving input images with filter-kernels for object recognition and classification purposes. In this paper, we propose a photonics circuit architecture which could consume a fraction of energy per inference compared with state of the art electronics.
△ Less
Submitted 16 August, 2018; v1 submitted 9 August, 2018;
originally announced August 2018.
-
Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs
Authors:
Li Jing,
Yichen Shen,
Tena Dubček,
John Peurifoy,
Scott Skirlo,
Yann LeCun,
Max Tegmark,
Marin Soljačić
Abstract:
Using unitary (instead of general) matrices in artificial neural networks (ANNs) is a promising way to solve the gradient explosion/vanishing problem, as well as to enable ANNs to learn long-term correlations in the data. This approach appears particularly promising for Recurrent Neural Networks (RNNs). In this work, we present a new architecture for implementing an Efficient Unitary Neural Networ…
▽ More
Using unitary (instead of general) matrices in artificial neural networks (ANNs) is a promising way to solve the gradient explosion/vanishing problem, as well as to enable ANNs to learn long-term correlations in the data. This approach appears particularly promising for Recurrent Neural Networks (RNNs). In this work, we present a new architecture for implementing an Efficient Unitary Neural Network (EUNNs); its main advantages can be summarized as follows. Firstly, the representation capacity of the unitary space in an EUNN is fully tunable, ranging from a subspace of SU(N) to the entire unitary space. Secondly, the computational complexity for training an EUNN is merely $\mathcal{O}(1)$ per parameter. Finally, we test the performance of EUNNs on the standard copying task, the pixel-permuted MNIST digit recognition benchmark as well as the Speech Prediction Test (TIMIT). We find that our architecture significantly outperforms both other state-of-the-art unitary RNNs and the LSTM architecture, in terms of the final performance and/or the wall-clock training speed. EUNNs are thus promising alternatives to RNNs and LSTMs for a wide variety of applications.
△ Less
Submitted 3 April, 2017; v1 submitted 15 December, 2016;
originally announced December 2016.
-
Binary matrices of optimal autocorrelations as alignment marks
Authors:
Scott A. Skirlo,
Ling Lu,
Marin Soljačić
Abstract:
We define a new class of binary matrices by maximizing the peak-sidelobe distances in the aperiodic autocorrelations. These matrices can be used as robust position marks for in-plane spatial alignment. The optimal square matrices of dimensions up to 7 by 7 and optimal diagonally-symmetric matrices of 8 by 8 and 9 by 9 were found by exhaustive searches.
We define a new class of binary matrices by maximizing the peak-sidelobe distances in the aperiodic autocorrelations. These matrices can be used as robust position marks for in-plane spatial alignment. The optimal square matrices of dimensions up to 7 by 7 and optimal diagonally-symmetric matrices of 8 by 8 and 9 by 9 were found by exhaustive searches.
△ Less
Submitted 28 August, 2014;
originally announced August 2014.
