-
FPGA Implementation of Low-Power Multiplierless Pre-Processing Free Chromatic Dispersion Equalizer
Authors:
Geraldo Gomes,
Pedro Freire,
Jaroslaw E. Prilepsky,
Sergei K. Turitsyn
Abstract:
We present a novel time-domain chromatic dispersion equalizer, implemented on FPGA, eliminating pre-processing and multipliers, achieving up to 54.3% energy savings over 80-1280 km with a simple, low-power design.
We present a novel time-domain chromatic dispersion equalizer, implemented on FPGA, eliminating pre-processing and multipliers, achieving up to 54.3% energy savings over 80-1280 km with a simple, low-power design.
△ Less
Submitted 23 December, 2024;
originally announced December 2024.
-
FPGA Implementation of Complex Value-based Clustering Filter for Chromatic Dispersion Compensation in Coherent Metro Links with Ultra-low Power Consumption
Authors:
Geraldo Gomes,
Pedro Freire,
Jaroslaw E. Prilepsky,
Sergei K. Turitsyn
Abstract:
This paper introduces a new machine learning-assisted chromatic dispersion compensation filter, demonstrating its superior power efficiency compared to conventional FFT-based filters for metro link distances. Validations on FPGA confirmed an energy efficiency gain of up to 63.5\% compared to the standard frequency-domain chromatic dispersion equalizer.
This paper introduces a new machine learning-assisted chromatic dispersion compensation filter, demonstrating its superior power efficiency compared to conventional FFT-based filters for metro link distances. Validations on FPGA confirmed an energy efficiency gain of up to 63.5\% compared to the standard frequency-domain chromatic dispersion equalizer.
△ Less
Submitted 20 September, 2024;
originally announced September 2024.
-
Geometric Clustering for Hardware-Efficient Implementation of Chromatic Dispersion Compensation
Authors:
Geraldo Gomes,
Pedro Freire,
Jaroslaw E. Prilepsky,
Sergei K. Turitsyn
Abstract:
Power efficiency remains a significant challenge in modern optical fiber communication systems, driving efforts to reduce the computational complexity of digital signal processing, particularly in chromatic dispersion compensation (CDC) algorithms. While various strategies for complexity reduction have been proposed, many lack the necessary hardware implementation to validate their benefits. This…
▽ More
Power efficiency remains a significant challenge in modern optical fiber communication systems, driving efforts to reduce the computational complexity of digital signal processing, particularly in chromatic dispersion compensation (CDC) algorithms. While various strategies for complexity reduction have been proposed, many lack the necessary hardware implementation to validate their benefits. This paper provides a theoretical analysis of the tap overlapping effect in CDC filters for coherent receivers, introduces a novel Time-Domain Clustered Equalizer (TDCE) technique based on this concept, and presents a Field-Programmable Gate Array (FPGA) implementation for validation. We developed an innovative parallelization method for TDCE, implementing it in hardware for fiber lengths up to 640 km. A fair comparison with the state-of-the-art frequency domain equalizer (FDE) under identical conditions is also conducted. Our findings highlight that implementation strategies, including parallelization and memory management, are as crucial as computational complexity in determining hardware complexity and energy efficiency. The proposed TDCE hardware implementation achieves up to 70.7\% energy savings and 71.4\% multiplier usage savings compared to FDE, despite its higher computational complexity.
△ Less
Submitted 16 September, 2024;
originally announced September 2024.
-
Multi-Task Learning to Enhance Generalizability of Neural Network Equalizers in Coherent Optical Systems
Authors:
Sasipim Srivallapanondh,
Pedro J. Freire,
Ashraful Alam,
Nelson Costa,
Bernhard Spinnler,
Antonio Napoli,
Egor Sedov,
Sergei K. Turitsyn,
Jaroslaw E. Prilepsky
Abstract:
For the first time, multi-task learning is proposed to improve the flexibility of NN-based equalizers in coherent systems. A "single" NN-based equalizer improves Q-factor by up to 4 dB compared to CDC, without re-training, even with variations in launch power, symbol rate, or transmission distance.
For the first time, multi-task learning is proposed to improve the flexibility of NN-based equalizers in coherent systems. A "single" NN-based equalizer improves Q-factor by up to 4 dB compared to CDC, without re-training, even with variations in launch power, symbol rate, or transmission distance.
△ Less
Submitted 3 November, 2023; v1 submitted 4 July, 2023;
originally announced July 2023.
-
Implementing Neural Network-Based Equalizers in a Coherent Optical Transmission System Using Field-Programmable Gate Arrays
Authors:
Pedro J. Freire,
Sasipim Srivallapanondh,
Michael Anderson,
Bernhard Spinnler,
Thomas Bex,
Tobias A. Eriksson,
Antonio Napoli,
Wolfgang Schairer,
Nelson Costa,
Michaela Blott,
Sergei K. Turitsyn,
Jaroslaw E. Prilepsky
Abstract:
In this work, we demonstrate the offline FPGA realization of both recurrent and feedforward neural network (NN)-based equalizers for nonlinearity compensation in coherent optical transmission systems. First, we present a realization pipeline showing the conversion of the models from Python libraries to the FPGA chip synthesis and implementation. Then, we review the main alternatives for the hardwa…
▽ More
In this work, we demonstrate the offline FPGA realization of both recurrent and feedforward neural network (NN)-based equalizers for nonlinearity compensation in coherent optical transmission systems. First, we present a realization pipeline showing the conversion of the models from Python libraries to the FPGA chip synthesis and implementation. Then, we review the main alternatives for the hardware implementation of nonlinear activation functions. The main results are divided into three parts: a performance comparison, an analysis of how activation functions are implemented, and a report on the complexity of the hardware. The performance in Q-factor is presented for the cases of bidirectional long-short-term memory coupled with convolutional NN (biLSTM + CNN) equalizer, CNN equalizer, and standard 1-StpS digital back-propagation (DBP) for the simulation and experiment propagation of a single channel dual-polarization (SC-DP) 16QAM at 34 GBd along 17x70km of LEAF. The biLSTM+CNN equalizer provides a similar result to DBP and a 1.7 dB Q-factor gain compared with the chromatic dispersion compensation baseline in the experimental dataset. After that, we assess the Q-factor and the impact of hardware utilization when approximating the activation functions of NN using Taylor series, piecewise linear, and look-up table (LUT) approximations. We also show how to mitigate the approximation errors with extra training and provide some insights into possible gradient problems in the LUT approximation. Finally, to evaluate the complexity of hardware implementation to achieve 200G and 400G throughput, fixed-point NN-based equalizers with approximated activation functions are developed and implemented in an FPGA.
△ Less
Submitted 19 February, 2023; v1 submitted 9 December, 2022;
originally announced December 2022.
-
Knowledge Distillation Applied to Optical Channel Equalization: Solving the Parallelization Problem of Recurrent Connection
Authors:
Sasipim Srivallapanondh,
Pedro J. Freire,
Bernhard Spinnler,
Nelson Costa,
Antonio Napoli,
Sergei K. Turitsyn,
Jaroslaw E. Prilepsky
Abstract:
To circumvent the non-parallelizability of recurrent neural network-based equalizers, we propose knowledge distillation to recast the RNN into a parallelizable feedforward structure. The latter shows 38\% latency decrease, while impacting the Q-factor by only 0.5dB.
To circumvent the non-parallelizability of recurrent neural network-based equalizers, we propose knowledge distillation to recast the RNN into a parallelizable feedforward structure. The latter shows 38\% latency decrease, while impacting the Q-factor by only 0.5dB.
△ Less
Submitted 8 December, 2022;
originally announced December 2022.
-
Reducing Computational Complexity of Neural Networks in Optical Channel Equalization: From Concepts to Implementation
Authors:
Pedro J. Freire,
Antonio Napoli,
Diego Arguello Ron,
Bernhard Spinnler,
Michael Anderson,
Wolfgang Schairer,
Thomas Bex,
Nelson Costa,
Sergei K. Turitsyn,
Jaroslaw E. Prilepsky
Abstract:
In this paper, a new methodology is proposed that allows for the low-complexity development of neural network (NN) based equalizers for the mitigation of impairments in high-speed coherent optical transmission systems. In this work, we provide a comprehensive description and comparison of various deep model compression approaches that have been applied to feed-forward and recurrent NN designs. Add…
▽ More
In this paper, a new methodology is proposed that allows for the low-complexity development of neural network (NN) based equalizers for the mitigation of impairments in high-speed coherent optical transmission systems. In this work, we provide a comprehensive description and comparison of various deep model compression approaches that have been applied to feed-forward and recurrent NN designs. Additionally, we evaluate the influence these strategies have on the performance of each NN equalizer. Quantization, weight clustering, pruning, and other cutting-edge strategies for model compression are taken into consideration. In this work, we propose and evaluate a Bayesian optimization-assisted compression, in which the hyperparameters of the compression are chosen to simultaneously reduce complexity and improve performance. In conclusion, the trade-off between the complexity of each compression approach and its performance is evaluated by utilizing both simulated and experimental data in order to complete the analysis. By utilizing optimal compression approaches, we show that it is possible to design an NN-based equalizer that is simpler to implement and has better performance than the conventional digital back-propagation (DBP) equalizer with only one step per span. This is accomplished by reducing the number of multipliers used in the NN equalizer after applying the weighted clustering and pruning algorithms. Furthermore, we demonstrate that an equalizer based on NN can also achieve superior performance while still maintaining the same degree of complexity as the full electronic chromatic dispersion compensation block. We conclude our analysis by highlighting open questions and existing challenges, as well as possible future research directions.
△ Less
Submitted 26 November, 2022; v1 submitted 26 August, 2022;
originally announced August 2022.
-
Computational Complexity Evaluation of Neural Network Applications in Signal Processing
Authors:
Pedro Freire,
Sasipim Srivallapanondh,
Antonio Napoli,
Jaroslaw E. Prilepsky,
Sergei K. Turitsyn
Abstract:
In this paper, we provide a systematic approach for assessing and comparing the computational complexity of neural network layers in digital signal processing. We provide and link four software-to-hardware complexity measures, defining how the different complexity metrics relate to the layers' hyper-parameters. This paper explains how to compute these four metrics for feed-forward and recurrent la…
▽ More
In this paper, we provide a systematic approach for assessing and comparing the computational complexity of neural network layers in digital signal processing. We provide and link four software-to-hardware complexity measures, defining how the different complexity metrics relate to the layers' hyper-parameters. This paper explains how to compute these four metrics for feed-forward and recurrent layers, and defines in which case we ought to use a particular metric depending on whether we characterize a more soft- or hardware-oriented application. One of the four metrics, called `the number of additions and bit shifts (NABS)', is newly introduced for heterogeneous quantization. NABS characterizes the impact of not only the bitwidth used in the operation but also the type of quantization used in the arithmetical operations. We intend this work to serve as a baseline for the different levels (purposes) of complexity estimation related to the neural networks' application in real-time digital signal processing, aiming at unifying the computational complexity estimation.
△ Less
Submitted 10 March, 2024; v1 submitted 24 June, 2022;
originally announced June 2022.
-
Towards FPGA Implementation of Neural Network-Based Nonlinearity Mitigation Equalizers in Coherent Optical Transmission Systems
Authors:
Pedro J. Freire,
Michael Anderson,
Bernhard Spinnler,
Thomas Bex,
Jaroslaw E. Prilepsky,
Tobias A. Eriksson,
Nelson Costa,
Wolfgang Schairer,
Michaela Blott,
Antonio Napoli,
Sergei K. Turitsyn
Abstract:
For the first time, recurrent and feedforward neural network-based equalizers for nonlinearity compensation are implemented in an FPGA, with a level of complexity comparable to that of a dispersion equalizer. We demonstrate that the NN-based equalizers can outperform a 1 step-per-span DBP.
For the first time, recurrent and feedforward neural network-based equalizers for nonlinearity compensation are implemented in an FPGA, with a level of complexity comparable to that of a dispersion equalizer. We demonstrate that the NN-based equalizers can outperform a 1 step-per-span DBP.
△ Less
Submitted 24 June, 2022;
originally announced June 2022.
-
Model-Based Deep Learning of Joint Probabilistic and Geometric Shaping for Optical Communication
Authors:
Vladislav Neskorniuk,
Andrea Carnio,
Domenico Marsella,
Sergei K. Turitsyn,
Jaroslaw E. Prilepsky,
Vahid Aref
Abstract:
Autoencoder-based deep learning is applied to jointly optimize geometric and probabilistic constellation shaping for optical coherent communication. The optimized constellation shaping outperforms the 256 QAM Maxwell-Boltzmann probabilistic distribution with extra 0.05 bits/4D-symbol mutual information for 64 GBd transmission over 170 km SMF link.
Autoencoder-based deep learning is applied to jointly optimize geometric and probabilistic constellation shaping for optical coherent communication. The optimized constellation shaping outperforms the 256 QAM Maxwell-Boltzmann probabilistic distribution with extra 0.05 bits/4D-symbol mutual information for 64 GBd transmission over 170 km SMF link.
△ Less
Submitted 5 April, 2022;
originally announced April 2022.
-
Domain Adaptation: the Key Enabler of Neural Network Equalizers in Coherent Optical Systems
Authors:
Pedro J. Freire,
Bernhard Spinnler,
Daniel Abode,
Jaroslaw E. Prilepsky,
Abdallah A. I. Ali,
Nelson Costa,
Wolfgang Schairer,
Antonio Napoli,
Andrew D. Ellis,
Sergei K. Turitsyn
Abstract:
We introduce the domain adaptation and randomization approach for calibrating neural network-based equalizers for real transmissions, using synthetic data. The approach renders up to 99\% training process reduction, which we demonstrate in three experimental setups.
We introduce the domain adaptation and randomization approach for calibrating neural network-based equalizers for real transmissions, using synthetic data. The approach renders up to 99\% training process reduction, which we demonstrate in three experimental setups.
△ Less
Submitted 25 February, 2022;
originally announced February 2022.
-
Neural Networks-based Equalizers for Coherent Optical Transmission: Caveats and Pitfalls
Authors:
Pedro J. Freire,
Antonio Napoli,
Bernhard Spinnler,
Nelson Costa,
Sergei K. Turitsyn,
Jaroslaw E. Prilepsky
Abstract:
This paper performs a detailed, multi-faceted analysis of key challenges and common design caveats related to the development of efficient neural networks (NN) nonlinear channel equalizers in coherent optical communication systems. Our study aims to guide researchers and engineers working in this field. We start by clarifying the metrics used to evaluate the equalizers' performance, relating them…
▽ More
This paper performs a detailed, multi-faceted analysis of key challenges and common design caveats related to the development of efficient neural networks (NN) nonlinear channel equalizers in coherent optical communication systems. Our study aims to guide researchers and engineers working in this field. We start by clarifying the metrics used to evaluate the equalizers' performance, relating them to the loss functions employed in the training of the NN equalizers. The relationships between the channel propagation model's accuracy and the performance of the equalizers are addressed and quantified. Next, we assess the impact of the order of the pseudo-random bit sequence used to generate the -- numerical and experimental -- data as well as of the DAC memory limitations on the operation of the NN equalizers both during the training and validation phases. Finally, we examine the critical issues of overfitting limitations, the difference between using classification instead of regression, and batch-size-related peculiarities. We conclude by providing analytical expressions for the equalizers' complexity evaluation in the digital signal processing (DSP) terms and relate the metrics to the processing latency.
△ Less
Submitted 31 May, 2022; v1 submitted 30 September, 2021;
originally announced September 2021.
-
Deep Neural Network-aided Soft-Demapping in Optical Coherent Systems: Regression versus Classification
Authors:
Pedro J. Freire,
Jaroslaw E. Prilepsky,
Yevhenii Osadchuk,
Sergei K. Turitsyn,
Vahid Aref
Abstract:
We examine here what type of predictive modelling, classification, or regression, using neural networks (NN), fits better the task of soft-demapping based post-processing in coherent optical communications, where the transmission channel is nonlinear and dispersive. For the first time, we present possible drawbacks in using each type of predictive task in a machine learning context, considering th…
▽ More
We examine here what type of predictive modelling, classification, or regression, using neural networks (NN), fits better the task of soft-demapping based post-processing in coherent optical communications, where the transmission channel is nonlinear and dispersive. For the first time, we present possible drawbacks in using each type of predictive task in a machine learning context, considering the nonlinear coherent optical channel equalization/soft-demapping problem. We study two types of equalizers based on the feed-forward and recurrent NNs, for several transmission scenarios, in linear and nonlinear regimes of the optical channel. We point out that even though from the information theory perspective the cross-entropy loss (classification) is the most suitable option for our problem, the NN models based on the cross-entropy loss function can severely suffer from learning problems. The latter translates into the fact that regression-based learning is typically superior in terms of delivering higher Q-factor and achievable information rates. In short, we show by empirical evidence that loss functions based on cross-entropy may not be necessarily the most suitable option for training communication systems in practical scenarios when overfitting- and vanishing gradients-related problems come into play.
△ Less
Submitted 22 August, 2022; v1 submitted 28 September, 2021;
originally announced September 2021.
-
Experimental Evaluation of Computational Complexity for Different Neural Network Equalizers in Optical Communications
Authors:
Pedro J. Freire,
Yevhenii Osadchuk,
Antonio Napoli,
Bernhard Spinnler,
Wolfgang Schairer,
Nelson Costa,
Jaroslaw E. Prilepsky,
Sergei K. Turitsyn
Abstract:
Addressing the neural network-based optical channel equalizers, we quantify the trade-off between their performance and complexity by carrying out the comparative analysis of several neural network architectures, presenting the results for TWC and SSMF set-ups.
Addressing the neural network-based optical channel equalizers, we quantify the trade-off between their performance and complexity by carrying out the comparative analysis of several neural network architectures, presenting the results for TWC and SSMF set-ups.
△ Less
Submitted 17 September, 2021;
originally announced September 2021.
-
Experimental implementation of a neural network optical channel equalizer in restricted hardware using pruning and quantization
Authors:
Diego R. Arguello,
Pedro J. Freire,
Jaroslaw E. Prilepsky,
Antonio Napoli,
Morteza Kamalian-Kopae,
Sergei K. Turitsyn
Abstract:
The deployment of artificial neural networks-based optical channel equalizers on edge-computing devices is critically important for the next generation of optical communication systems. However, this is still a highly challenging problem, mainly due to the computational complexity of the artificial neural networks (NNs) required for the efficient equalization of nonlinear optical channels with lar…
▽ More
The deployment of artificial neural networks-based optical channel equalizers on edge-computing devices is critically important for the next generation of optical communication systems. However, this is still a highly challenging problem, mainly due to the computational complexity of the artificial neural networks (NNs) required for the efficient equalization of nonlinear optical channels with large dispersion-induced memory. To implement the NN-based optical channel equalizer in hardware, a substantial complexity reduction is needed, while we have to keep an acceptable performance level of the simplified NN model. In this work, we address the complexity reduction problem by applying pruning and quantization techniques to an NN-based optical channel equalizer. We use an exemplary NN architecture, the multi-layer perceptron (MLP), to mitigate the impairments for 30GBd 1000km transmission over a standard single-mode fiber, and demonstrate that it is feasible to reduce the equalizer's memory by up to 87.12%, and its complexity by up to 78.34%, without noticeable performance degradation. In addition to this, we accurately define the computational complexity of a compressed NN-based equalizer in the digital signal processing (DSP) sense. Further, we examine the impact of using different CPU and GPU settings on the power consumption and latency for the compressed equalizer. We also verify the developed technique experimentally, by implementing the reduced NN equalizer on two standard edge-computing hardware units: Raspberry Pi 4 and Nvidia Jetson Nano, which are used to process the data generated via simulating the signal's propagation down the optical-fiber system.
△ Less
Submitted 12 March, 2022; v1 submitted 15 September, 2021;
originally announced September 2021.
-
End-to-End Deep Learning of Long-Haul Coherent Optical Fiber Communications via Regular Perturbation Model
Authors:
Vladislav Neskorniuk,
Andrea Carnio,
Vinod Bajaj,
Domenico Marsella,
Sergei K. Turitsyn,
Jaroslaw E. Prilepsky,
Vahid Aref
Abstract:
We present a novel end-to-end autoencoder-based learning for coherent optical communications using a "parallelizable" perturbative channel model. We jointly optimized constellation shaping and nonlinear pre-emphasis achieving mutual information gain of 0.18 bits/sym./pol. simulating 64 GBd dual-polarization single-channel transmission over 30x80 km G.652 SMF link with EDFAs.
We present a novel end-to-end autoencoder-based learning for coherent optical communications using a "parallelizable" perturbative channel model. We jointly optimized constellation shaping and nonlinear pre-emphasis achieving mutual information gain of 0.18 bits/sym./pol. simulating 64 GBd dual-polarization single-channel transmission over 30x80 km G.652 SMF link with EDFAs.
△ Less
Submitted 26 July, 2021;
originally announced July 2021.
-
Power and Modulation Format Transfer Learning for Neural Network Equalizers in Coherent Optical Transmission Systems
Authors:
Pedro J. Freire,
Daniel Abode,
Jaroslaw E. Prilepsky,
Sergei K. Turitsyn
Abstract:
Transfer learning is proposed to adapt an NN-based nonlinear equalizer across different launch powers and modulation formats using a 450km TWC-fiber transmission. The result shows up to 92% reduction in epochs or 90% in the training dataset.
Transfer learning is proposed to adapt an NN-based nonlinear equalizer across different launch powers and modulation formats using a 450km TWC-fiber transmission. The result shows up to 92% reduction in epochs or 90% in the training dataset.
△ Less
Submitted 24 June, 2021;
originally announced June 2021.
-
Experimental Study of Deep Neural Network Equalizers Performance in Optical Links
Authors:
Pedro J. Freire,
Yevhenii Osadchuk,
Bernhard Spinnler,
Wolfgang Schairer,
Antonio Napoli,
Nelson Costa,
Jaroslaw E. Prilepsky,
Sergei K. Turitsyn
Abstract:
We propose a convolutional-recurrent channel equalizer and experimentally demonstrate 1dB Q-factor improvement both in single-channel and 96 x WDM, DP-16QAM transmission over 450km of TWC fiber. The new equalizer outperforms previous NN-based approaches and a 3-steps-per-span DBP.
We propose a convolutional-recurrent channel equalizer and experimentally demonstrate 1dB Q-factor improvement both in single-channel and 96 x WDM, DP-16QAM transmission over 450km of TWC fiber. The new equalizer outperforms previous NN-based approaches and a 3-steps-per-span DBP.
△ Less
Submitted 24 June, 2021;
originally announced June 2021.
-
Transfer Learning for Neural Networks-based Equalizers in Coherent Optical Systems
Authors:
Pedro J. Freire,
Daniel Abode,
Jaroslaw E. Prilepsky,
Nelson Costa,
Bernhard Spinnler,
Antonio Napoli,
Sergei K. Turitsyn
Abstract:
In this work, we address the question of the adaptability of artificial neural networks (NNs) used for impairments mitigation in optical transmission systems. We demonstrate that by using well-developed techniques based on the concept of transfer learning, we can efficaciously retrain NN-based equalizers to adapt to the changes in the transmission system, using just a fraction (down to 1%) of the…
▽ More
In this work, we address the question of the adaptability of artificial neural networks (NNs) used for impairments mitigation in optical transmission systems. We demonstrate that by using well-developed techniques based on the concept of transfer learning, we can efficaciously retrain NN-based equalizers to adapt to the changes in the transmission system, using just a fraction (down to 1%) of the initial training data or epochs. We evaluate the capability of transfer learning to adapt the NN to changes in the launch power, modulation format, symbol rate, or even fiber plants (different fiber types and lengths). The numerical examples utilize the recently introduced NN equalizer consisting of a convolutional layer coupled with bi-directional long-short term memory (biLSTM) recurrent NN element. Our analysis focuses on long-haul coherent optical transmission systems for two types of fibers: the standard single-mode fiber (SSMF) and the TrueWave Classic (TWC) fiber. We underline the specific peculiarities that occur when transferring the learning in coherent optical communication systems and draw the limits for the transfer learning efficiency. Our results demonstrate the effectiveness of transfer learning for the fast adaptation of NN architectures to different transmission regimes and scenarios, paving the way for engineering flexible and universal solutions for nonlinearity mitigation.
△ Less
Submitted 21 September, 2021; v1 submitted 11 April, 2021;
originally announced April 2021.
-
Performance versus Complexity Study of Neural Network Equalizers in Coherent Optical Systems
Authors:
Pedro J. Freire,
Yevhenii Osadchuk,
Bernhard Spinnler,
Antonio Napoli,
Wolfgang Schairer,
Nelson Costa,
Jaroslaw E. Prilepsky,
Sergei K. Turitsyn
Abstract:
We present the results of the comparative analysis of the performance versus complexity for several types of artificial neural networks (NNs) used for nonlinear channel equalization in coherent optical communication systems. The comparison has been carried out using an experimental set-up with transmission dominated by the Kerr nonlinearity and component imperfections. For the first time, we inves…
▽ More
We present the results of the comparative analysis of the performance versus complexity for several types of artificial neural networks (NNs) used for nonlinear channel equalization in coherent optical communication systems. The comparison has been carried out using an experimental set-up with transmission dominated by the Kerr nonlinearity and component imperfections. For the first time, we investigate the application to the channel equalization of the convolution layer (CNN) in combination with a bidirectional long short-term memory (biLSTM) layer and the design combining CNN with a multi-layer perceptron. Their performance is compared with the one delivered by the previously proposed NN equalizer models: one biLSTM layer, three-dense-layer perceptron, and the echo state network. Importantly, all architectures have been initially optimized by a Bayesian optimizer. We present the derivation of the computational complexity associated with each NN type -- in terms of real multiplications per symbol so that these results can be applied to a large number of communication systems. We demonstrated that in the specific considered experimental system the convolutional layer coupled with the biLSTM (CNN+biLSTM) provides the highest Q-factor improvement compared to the reference linear chromatic dispersion compensation (2.9 dB improvement). We examine the trade-off between the computational complexity and performance of all equalizers and demonstrate that the CNN+biLSTM is the best option when the computational complexity is not constrained, while when we restrict the complexity to lower levels, the three-layer perceptron provides the best performance. Our complexity analysis for different NNs is generic and can be applied in a wide range of physical and engineering systems.
△ Less
Submitted 23 June, 2021; v1 submitted 15 March, 2021;
originally announced March 2021.
