Eurotech: Securing edge AI with containers
Eurotech in Italy is taking the challenge of AI security down to the edge with software container technology.
The company won the award for AI and learning hardware at the recent Embedded World exhibition with the ReliaCOR 44-11, as Marco Carrer, CTO at Eurotech (above) explains
“We have designed 44-11 industrial PC with workload consolidation in mind as the IoT and AI come together,” said Carrer. “Even if it running at the edge an industrial PC has to be managed and device management is a typical IoT problem,” he tells eeNews Europe.
The 4411 uses either an Intel 13th Gen i7 or i9 processor with an Nvidia GPU. “We understand that one of the challenges of edge AI is choosing the right device so we have a portfolio with a fanless embedded version with the Nvidia Orin processor, then Orin AGX for higher performance that is picking up applications.”
The box bypasses the need for complex coding, enabling a quick transition from initial concept to full operation, and allows multiple models to be loaded and tested in sequence with capabilities for monitoring, managing, and updating models efficiently.
But it is the security of the hardware and software that is key, enabled by software containers..
“We now have the hardware with the ISO 62443-4-2 L2 security certification as new hardware may have new attack vectors and we are the first industrial PC with that listing. Our middleware acts as an agent and an application framework that has become an orchestrator for container workloads.”
This is a ‘ready to use’ AI box with Ubuntu Linux supported for 10 years and security, but what makes it the AI box is the framework for containers. This uses DeepStream, the Gstreamer version of Nvidia with drivers for every camera that connects to the box. This allows the camera to be configured by simply selecting the frame rate in the container.
However for edge systems the box cam be handling multiple cameras.
“The question is how do you make the data flow as fast as possible without redundancy,” he says. One technique supported by the 11-44 is to use the MQTT IoT protocol and Nvidia’s Triton AI software framework.
“The container grabs the image and the MQTT broker acts as a sink for notification and we have Nvidia Triton pre-installed and this determines what models are running for object detection, defect detection, with an output to a digital twin, a data logger, to the cloud, then you have the IoT data. We use that as the actuator for a robot arm for example to dispose of defective parts.”
“In terms of 6443 you have root file system encryption with secure boot that unlocks the file system. The model as it is distributed form the cloud to the edge is encrypted and we can use the orchestrator as a security element that loads the decrypts the model in RAM at runtime.”
“Right now we want to make sure there is no at rest unencrypted data. The frames go directly through RAM. We are looking at kernel RAM encryption and I think that will come for Level 3 6443 certification.”
“One of the last pieces is a zero touch provision for AWS so we use the TPM (Trusted Platform Module) as a device identity provider and by using certificate management on the first activation we create short lived operational identifies in the TPM. This means if the system is compromised it cannot be tampered with,” he said.
Nvidia is developing a system on chip called Thor using the Blackwell GPU architecture that will be used for Jetson cards. However the additional performance in AI comes from moving from the 8bit integer mode (INT8) to 4bit (INT4).
“Thor is promising 20 to 50% more performance on certain workloads and if the module is pin to pin compatible we can upgrade the systems in no time. Nvidia makes it pretty easy for the quantization from INT8 to INT4, it would require rebuilding the model so from that perspective I’m quite positive.”
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
