So how did the GPU companies succeed with all these tough challenges and hurdles? What they did really well is convince videogame developers that the GPU companies were expanding the market for videogames. That is what AI processor companies need to think about: how can AI software developers invest in their platforms and grow the AI software market?
So what are lessons from the videogames industry?
The GPU companies never said their GPUs were faster or cheaper than CPUs, they said you could create better graphics. This needs to be the message for new AI accelerators: saying you can do AI faster isn’t that interesting, because NVIDIA has already achieved massive performance with AI. You can’t easily beat NVIDIA on price, because even though the per-unit cost of an AI GPU may be high, the up-front investment in porting software adds a lot to the total cost. An AI accelerator needs to enable better, more complex, more reliable AI than is possible with a GPU. Higher performance per Watt helps you get there, but it’s a means to an end, not an end itself. There’s lots that is possible here: look how Tesla took various IP blocks and customized them with their own IP for their FSD chip, all designed around their existing AI software.
The GPU companies knew that software developers were loss-leaders, but they needed to attract them in order to bring in the game developers. Therefore, AI processor companies should be practically giving away their processors to software developers and academics to get the software ported.
The main AI software frameworks should be constantly testing on these new AI processors, with the results available publicly. Yes, this will expose performance differences and bugs, but that would be a good thing: we’re only going to fix the problems if people can see the problems to fix.
For autonomous vehicles, we really need to work on safety. This is a massive challenge that needs to go from the bottom up. Not just theoretical discussions about the age-old “trolley problem”, but the real engineering challenges of delivering safe AI software.
We need to build up an open ecosystem of software to rival NVIDIA’s. The only way of enabling this is with open standards. In graphics, this happened with OpenGL and then Vulkan, which is a recent graphics standard designed by, and for the, game developers. We need to see how we can do this with AI software developers. This is a challenge since graphics software is an established field and AI software is not. There are a lot of conflicting opinions but, if we don’t then NVIDIA’s CUDA lock-in will easily control the AI software market for decades.
We need to work together at this as an industry. There are no easy answers, mainly because this is pioneering work. Like any pioneers, we need to help each other to overcome the huge challenges. If we stay isolated, on our own, not speaking to each other, then every new AI challenger will not be welcomed by developers or customers.
If you want to be part of this open ecosystem, there are many ways you can get involved. Join the standards bodies: MISRA (for safety), Khronos (for the acceleration standards) or ISO C++ (for accelerating general programming). Become part of open-source projects that operate on open standards for AI, whether that’s SYCL, OpenCL or SPIR-V. It’s easy to port existing AI projects from NVIDIA’s CUDA to the SYCL open standard, or to port graph-compiler projects from NVIDIA’s PTX to the open standard SPIR-V. We can achieve so much more by working together.
Andrew Richards is CEO and co-founder of Codeplay Software Ltd. (Edinburgh, Scotland) a pioneer in GPU acceleration.
Richards started his career writing video games in the days of 8bit computers, progressing to become a lead games programmer at Eutechnyx, where he wrote best-selling titles such as Pete Sampras Tennis and Total Drivin’. Andrew developed early GPU compiler technology, and founded Codeplay in 2002. Codeplay has been producing compilers for games consoles, for special-purpose processors and for GPUs ever since.
Richards is also the chair of the software working group of the HSA Foundation and former chair of the SYCL for OpenCL sub-group of the Khronos Group.
Codeplay is now working on artificial intelligence and safety for self-driving cars.
Related links and articles: