First industry standard real world AI benchmark
The Transaction Processing Performance Council (TPC) has launched the first industry-standard, vendor-neutral benchmark for measuring real-world, end-to-end AI and machine learning (ML) scenarios.
The TPCx-AI benchmarks uses a diverse dataset and was specifically designed to be adaptable across a wide range of different systems from the edge to the data centre. This should allow a comparison of edge AI chips, and data centre chips and FPGA acclerators for real world applications.
“The TPCx-AI benchmark is the result of collaboration between talented engineers and researchers at some of today’s leading AI organizations,” said Hamesh Patel, chair of the TPCx-AI committee and principal engineer at Intel Corporation. “It is designed to emulate real-world examples of organizations that use a variety of production ready data science pipelines – including both AI and ML approaches – and is now widely available to anyone who would like to download and run it. We look forward to feedback as industry experts, academics and others interested in benchmarking system performance begin to use it.”
The TPCx-AI benchmark provides a means to evaluate performance for the System Under Test (SUT) as a general-purpose data science system that generates and processes large volumes of data and trains preprocessed data to produce realistic machine learning models. The benchmark allows for flexibility in configuration changes.
The benchmark was developed by a group that includes Intel, IBM, Cisco, Dell, HPE, Microsoft, Red Hat, TTA, and VMware.
The benchmark also measures the end-to-end time to provide insights for individual use cases, as well as throughput metrics to simulate multiuser environments for a given hardware, operating system, and data processing system configuration under a controlled, complex, multi-user AI or machine learning data science workload.
Other members of the TPC include Actian, Alibaba, AMD, Fujitsu, Hitachi, Huawei, Lenovo, Nvidia and Oracle.
As an “Express” benchmark, TPCx-AI is an executable kit that can be rapidly deployed and measured. It is designed to provide relevant, objective performance data to industry users and is available for download at
https://tpc.org/tpcx-ai/default5.asp; https://tpc.org/information/about/join5.asp
Related articles
- Baidu releases AI benchmark (2016)
- Edge startup offers legacy support and ML acceleration
- Mipsology takes on GPUs with AI on FPGAs
- Battle for the AI toolflow
Other articles on eenNews Europe
- European Chips Act for semiconductor sovereignty
- Apple launches the 1Tbyte edge iPhone 13
- Report calls for European FPGA tools to boost open source hardware
- Galway in line for Intel megafab
- TI to add machine learning to codeless brushless motor drivers
- Return of smartphone manufacturing in Germany
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
