At the current state of the art, test vehicles that enter data for the training of autonomous cars generate around 70 terabytes of data per day. Processing these huge amounts of data requires highly efficient processes in order to optimize vehicle operation and develop new functions. The Hybrid Data Bridge is intended to provide ZF with continuous, company-wide process control.
A pilot project of the Hybrid Data Bridge has been implemented and is now being used in the ADAS research and development department; further ZF divisions are to follow. It is planned that more than 25,000 ZF engineers worldwide will be able to access the Hybrid Data Bridge via an intuitive interface, for example to view individual video clips, label them, create playlists for training the ADAS/AD functions in the vehicle, and quickly transfer them to downstream processes.
As a central component of ZF’s hybrid cloud strategy, the Hybrid Data Bridge is based on complete virtualization of the vehicle’s computing resources. It comprises the services of various providers in a public and private cloud. The cloud architecture is based on the hybrid cloud foundation RedHat OpenShift for the creation and scaling of containerized applications, for example to accelerate data reception and video transformation.
“Managing huge amounts of data in a hybrid multi-cloud environment is very important. Transparent access to files in data lakes with low latency is essential, for example when developing autonomous vehicles, where we have to process images and information from many different data sources such as cameras, radar, lidar and communication systems to provide a complete picture of the situation inside and outside the vehicle,” explains Harald Holder, Director of IT Infrastructure Platforms at ZF. “Thanks to the intelligent connection with the data transfer solution IBM Aspera, these data volumes can be transferred quickly and securely and synchronized company-wide.
The Hybrid Data Bridge enables transfer, storage, and archiving of large amounts of data. The heart of the system is the IBM Archive and Essence Manager (AREMA) to manage complex video assets and large digital archives using Artificial Intelligence.
IBM Spectrum Scale is used to store the data. The scalable, high-performance solution for data and file management offers integrated tools for the information lifecycle that manage petabytes of data and billions of files. Depending on the speed and cost of data access, you can choose between flash, disk, tape or cloud object storage.