Real-time 360º video stitching: 3D next

April 19, 2017 // By Julien Happich
Last year, Cambridge-based Argon Design was demonstrating its Argon360 real-time 360º video stitching technology on an FPGA development platform (taking the video output of six Gopro cameras mounted on a cube).

More recently, the company has signed up its first licensee for this hardware-based intellectual property and is in discussion with a number of other chip integrators.

Discussing the merits of hardware-accelerated 360º video stitching with eeNews Europe, Argon Design's CTO and co-founder Steve Barlow put Argon360 in perspective.

"There are four different types of solutions for video stitching. The first one is in the cloud, off-line. You have to take the individual video streams and upload them separately. This is not done in real time and you can't see what you are creating, but it gives you a lot of processing power at low cost.

The second option is to use a GPU, you get nearly real time performance, with a delay of a few seconds but you don't get such a good stitching. A third solution is to use an FPGA to perform simple colour blending. Basically you computationally warp the images, overlay them and blend them.

We use multiband blending which is algorithmically more complicated but we can smooth out the slight differences between the cameras due to the different camera sensors, the discontinuity between the optics and the parallax. The technique we use smooths out the distance differences so we've got a good real time stitching solution with a latency of just one frame, with a power consumption about the same as a video encoder".

"Our target is below 0.5W on a modern process, 28nm or below", Barlow added.

Argon360 can handle any combination of up to six sensors with an output resolution of up to 8192x4320 at 30 frames per second and the company is targeting a number of market areas including security, action cameras, automotive electronic surround view applications and the professional 360º video immersive content creators for VR helmets. Drones represent another promising segment.

"With our very low latency, action cameras can directly produce 360º content and livestream it, users can preview the result straight away, and all this is feasible on a small battery-operated unit. There is no way you could rely on a GPU for this, it would be too power hungry and too slow", completed the CTO.

The company gets about 50% of its revenues from its consultancy services, helping other companies with board level, RTL, FPGA and embedded software design. The other 50% of revenues come from IP licensing of which Argon360 is the third offering.