For many AI applications 8 bit number crunching can do the job
HPC often requires the highest possible precision (i.e. 64-bit double precision floating point), while many AI applications actually work with 8-bit integers or floating point numbers
Waleed Atallah, Product Manager responsible for silicon, boards, and systems at Untether AI, gives us a look in the hardware world of AI and explains that a lot of AI applications doesn’t need more than 8 bit integers or floating point numbers.
In his article he explains what separates HPC from AI, they both do a lot of number crunching? The answer is in the precision required for a valid answer.
From the article:
The AI industry has begun coalescing around two preferred candidates for quantized data types: INT8 and FP8. Every hardware vendor seems to have taken a side. In mid 2022, a paper by Graphcore and AMD[2] floated the idea of an IEEE standard FP8 datatype. A subsequent joint paper with a similar proposal from Intel, Nvidia, and Arm[3] followed shortly. Other AI hardware vendors like Qualcomm[4, 5] and Untether AI[6] also wrote papers promoting FP8 and reviewing its merits versus INT8. But the debate is far from settled. While there is no singular answer for which data type is best for AI in general, there are superior and inferior data types when it comes to various AI processors and model architectures with specific performance and accuracy requirements.
Integer Versus Floating Point
Floating point and integer data types are two ways to represent and store numerical values in computer memory. There are a few key differences between the two formats that translate to advantages and disadvantages for various neural networks in training and inference.
Find his full article at HPCwire
Waleed Atallah is a Product Manager responsible for silicon, boards, and systems at Untether AI.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
