Why programming the GPU is going mainstream

I have spent this week at NVIDIA's GPU Technical Conference in San Jose. It has been thought-provoking for me; I had not realised the extent to which GPU programming is taking off.

By GPU programming, I do not mean creating games or cool graphics in DirectX or OpenGL - though in practice many applications involve some graphical elements - but using the GPU for general-purpose programming.

Why bother? Simply because for certain kinds of processing, the performance benefit is spectacular. It is easy to demonstrate. At the exhibition here, Jedox was demonstrating its Palo GPU Accelerator. The demo involves a business intelligence analysis in Excel that when running normally in Windows, takes several minutes to compute. Use the accelerator instead, and it is instant. Note that the accelerator is not normally on the same machine as Excel; in fact, you can have a GPU-based high performance server on a network and share it with multiple clients.

The technical reason for this high performance is that GPUs specialise in massively parallel processing. Whereas in CPUs havng 2, 4 or 8 cores on a system is common, an NVIDIA Tesla GPU starts at 240 cores, and you can put up to 8 in a single box. Teslas are GPUs that cannot drive a display; they are intended purely for GPU programming. You can also code against standard NVIDIA GPUs like Geforce. GPUs are not an alternative to CPUs, but as specialist number-crunching add-ins they are remarkable.

What is also interesting is that whereas CPU performance improvements have slowed thanks to technical issues such as limits on how far clock speed can be increased, GPU performance is still marching ahead. The parallel approach is more scalable, and because it does not depend on increasing clock speeds, more power-efficient. This means that the value added by GPU programming is likely to increase in the coming years.

The scientific, engineering and academic community has jumped on the GPU bandwagon since it brings significant benefits to their work. This has been evident at the GPU conference, where the floor is lined with posters summarising interesting GPU programming applications.

While the advantages of GPU programming are real, the supporting tools are still somewhat challenging. The most common approaches are either OpenCL, which works across GPUs from multiple vendors, or CUDA C, extensions to the C/C++ languages that let you execute code on the GPU and copy memory between the GPU and the main system, but which only works on NVIDIA GPUs. Although the OpenCL standard is attractive because it runs on more systems, CUDA C is widely used in scenarios where you can specify the hardware. Either way, you need to grok concurrent programming and the subtle bugs it can introduce. It will never be trivial, but there is work going on to bring GPU programming to more languages, such as Python and Microsoft .NET, as well as libraries to simplify managing memory across what is in effect a computer within a computer.

This week's conference has convinced me that GPU programming is something any developers need to be aware of. Where it goes from here will be interesting to watch. NVIDIA deserves credit for advancing the field, but faces competition from AMD, Intel and others. I would also expect the standards-based OpenCL approach to gain in popularity, though the word here is that it is not quite mature enough yet. Finally, NVIDIA is insistent that this is not just for the high-end, but that we will see GPU programming at every level from cloud to desktop to mobile device.