Backpropagation boost for deep learning in photonic neural networks
Researchers in Milan have built and trained a three layer neural network with backpropogation using silicon photonics as a key step to more power efficient machine learning.
Photonic neural networks efficiently transform optically encoded inputs using Mach-Zehnder interferometer mesh networks interleaved with nonlinearities. The team at the Politecnico di Milano (Polimi) trained the three-layer, four-port silicon photonic neural network with programmable phase shifters and optical power monitoring to solve classification tasks using in situ backpropagation, a photonic analog of the most popular method to train conventional neural networks.
The team measured backpropagated gradients for phase-shifter voltages by interfering forward- and backward-propagating light and simulated in situ backpropagation for 64-port photonic neural networks trained on MNIST image recognition given errors. All the experiments performed comparably to digital simulations with over 94% test accuracy and the energy scaling analysis indicated a route to scalable machine learning systems.
“An artificial neuron, like a biological neuron, must perform very simple mathematical operations, such as addition and multiplication, but in a neural network consisting of many densely interconnected neurons, the energy cost of these operations grows exponentially and quickly becomes prohibitive,” said Francesco Morichetti, Head of the Photonic Devices Lab at Polimi.
“Our chip incorporates a photonic accelerator that allows calculations to be carried out very quickly and efficiently, using a programmable grid of silicon interferometers. The calculation time is equal to the transit time of light in a chip a few millimetres in size, so we are talking about less than 0.1 nanoseconds,” he said.
“The advantages of photonic neural networks have long been known, but one of the missing pieces to fully exploit their potential was network training,” said Andrea Melloni, Director of Polifab, Polimi’s micro and nanotechnology centre. “It is like having a powerful calculator, but not knowing how to use it. In this study, we succeeded in implementing training strategies for photonic neurons similar to those used for conventional neural networks. The photonic “brain” learns quickly and accurately and can achieve precision comparable to that of a conventional neural network, but faster and with considerable energy savings. These are all building blocks for artificial intelligence and quantum applications.”
- Former Dialog CEO helps photonic AI startup raise $11.5m
- Photonics startup aims for world’s most powerful AI
In addition to neural networks, the device can be used as a computing unit for multiple applications where high computational efficiency is required such as graphics accelerators, mathematical coprocessors, data mining, cryptography and quantum computers.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
