Sander Olson Interview: Nvidias Sumit Gupta, Telsa GPU Computing Group Senior Product Manager

Here is an interview with NVIDIA’s Sumit Gupta. Dr. Gupta is a senior product manager with the NVIDIA Tesla GPU Computing Group. NVIDIA is heavily pushing GPU computing, and the field is expanding exponentially. There is an average of 10x-50x improvement using GPUs instead of CPUs, and they are finding that increasing numbers of applications can run dramatically faster with GPU computing.

Nvidia’s next GPGPU, GT300 will have 3 teraflops of performance.

Bill Daly, Nvidia’s chief scientist had an interview with a french website and indicated that Nvidia’s GPGPU’s should be at 20 teraflops by 2015.

Here is the Bill Daly interview translated to english on Babelfish.

Here is an interview with NVIDIA’s Sumit Gupta, Product Manager Tesla.

Question: How long has NVIDIA been working on scientific computing? What sort of resources is NVIDIA devoting to CUDA and GPU computing?

Answer: We have been working on high-performance computing since the 2003-2004 timeframe. The first hardware was unveiled in November 2006, and the first software became available in 2007.

Question: How many high-performance computing tasks can benefit from GPGPUs? What is the average speedup?

Answer: Practically any computing task that currently consumes substantial amounts of CPU time could benefit from GPU computing. We originally intended GPU computing for “embarrassingly parallel” computing tasks, but we have recently found that even chores that aren’t embarrassingly parallel can benefit substantially from GPU computing.

Question: Could you provide any specific examples?

Answer: Video encoding is not considered an embarrassingly parallel computing problem. However, video encoding can be substantially accelerated using GPUs, and there are several corporations that have announced video encoding using GPUs. One of those corporations is called Elemental Technologies. We have also discovered that sorting algorithms and many other tasks can be substantially enhanced by GPUs. Database applications are another excellent example. A fourth example is virus software scans, which can be performed quicker using a GPU. The GPU is a new architecture which isn’t constrained by the same limitations as x86 CPUs. So really, any number crunching application can take advantage of GPU computing.

Question: What is the average speedup of GPU-accelerated tasks?

Answer: It depends of course upon the application. For non-parallel tasks we are seeing a 12x speedup, but for applications that are embarrassingly parallel we are seeing much more impressive results. In instances where we replace a CPU with four GPUs we are seeing up to 270x improvements. On average we are seeing 10-50x improvements

Question: To what extent will the upcoming Windows7 and Snow Leopard operating systems take advantage of GPU computing?

Answer: Although I don’t know the details of how these upcoming operating systems will exploit GPUs, both OSs will work best with both a CPU and a GPU. OS developers now have a straightforward and easy way to exploit the capability of two processors – the CPU and GPU. Imaging and video processing are examples of the benefits of having both a CPU and GPU. Downloading and viewing photos, for instance, can and should be performed by the CPU. However, more complex tasks such as image enhancements and manipulations are extremely amenable to GPUs. As I mentioned earlier, virus scanning is another task that has traditionally been performed by the CPU but could actually benefit from GPUs – the scanner could use the GPU to perform continuous background scanning without any degradation in overall performance.

Question: It would seem that GPGPUs could transform the field of supercomputing.

Answer: There is already a GPU-based supercomputer that is the 29th fastest machine on the planet. That machine, called Tsubame, utilizes GPUs. The Tokyo Institute of Technology recently upgraded their Tsubame supercomputer with 170 of our 1u Tesla systems. In addition there is a growing list of cluster installations, such as the NCSA, Oak Ridge National Labs, several oil and gas companies, the National Taiwan University, and many others. Institutions such as these constantly need more supercomputing performance, so GPU computing is a huge benefit to them. It is inevitable that the GPU will become an integral part of supercomputing, and I predict that supercomputers employing both CPUs and GPUs will soon become the norm.

Question: What is the GPU ventures program?

Answer: The GPU ventures program was established by NVIDIA to fund startup corporations involved in GPU computing. Our goal is to nurture innovation in this field through direct investment and support. We have already made investments in corporations such as Acceleware, Elemental Technologies, Keyhole, and MotionDSP.

Question: What about data centers? Will CPU/GPU servers become common?

Answer: Yes – Supermicro has already announced the availability of GPU-optimized servers. These machines are CPU/GPU hybrids, containing a pair of Tesla GPUs. These systems provide several teraflops of single-precision performance and are highly scalable. Such systems are already in widespread use and will find their way into increasing numbers of data centers in the next few years. Essentially every OEM now offers GPU computing solutions. The list of data centers that is taking advantage of GPU computing is growing exponentially so I can’t provide precise numbers.

Question: Intel claims that their upcoming Larrabee processor will effectively obsolete GPGPUs.

Answer: I do not feel it is appropriate for me to comment on another company’s unreleased product. However, some of my colleagues here have noted some apparent drawbacks to Larrabee. They have noted, for instance, that Larrabee will use an x86 instruction set, which will impose inherent limitations and costs on the architecture. We welcome Intel’s endorsement of GPU computing and await to see what their new product looks like when it is finally available.

Question: Shouldn’t it be feasible to put a GPU and a CPU on the same die? Would there be advantages to such a system?

Answer: There are advantages but there are also many disadvantages to such an approach. CPU and GPUs have different design cycles, and the challenges of integrating both on a single IC are nontrivial. This perhaps explains why AMD/ATI is taking so long to get their Fusion products to market. Moreover, a CPU/GPU die will never be able to offer equivalent performance of a dedicated CPU coupled with a dedicated GPU.

Question: How many Universities teach CUDA? How many educational institutions are taking advantage of GPU computing?

Answer: There are currently over 200 Universities teaching CUDA, and the number is growing rapidly. This is a clear indication that this technology is reaching critical mass. We frequently receive calls from Universities informing us that they want to teach CUDA, OpenCL, and GPU computing. At this point, any institution teaching C programming should have no difficulty teaching CUDA or OpenCL. We actively support this process by providing CUDA software support, training material, and often by hardware donations as well.

Question: How will having access to GPU computing affect University computer programs?

Answer: Imagine a University student whose program requires 3 days computer time on a large CPU cluster. Now with CUDA they can run the same application in one day on a desktop with a NVIDIA GPU. For the 90% of students who don’t have ready access to a cluster, GPU computing could become their primary tool for running their computationally-intensive programs. For these people GPU computing is truly transformative.

Question: What role will GPGPUs play in cloud computing? Will it be possible to rent GPU time the way users can currently rent CPU time?

Answer: Absolutely – it is a natural extension to have GPUs in clouds for high-performance computing, and the cost/performance for GPU computing is sufficiently high to make most pure CPU clouds uncompetitive in the long run. But the visualization side of computing could also transition to the cloud. This would allow computers with limited graphics capability, such as mobile devices, to display sophisticated images and animations. Although I cannot go into detail, there are clouds that are now being created which are utilizing NVIDIA GPUs.

Question: How do you see GPGPU computing evolving over the next several years?

Answer: Within the next several years, GPU computing will continue to expand exponentially, and every application that can take advantage of a GPU will have to in order to remain competitive. The software industry has only recently become aware of this fact and is now expending considerable resources to take advantage of GPUs. In a couple of years users will have a choice for a certain price point of buying an 8- or 16-core CPU, or a 4-core CPU and a GPU. In the overwhelming majority of circumstances, the CPU/GPU combination will be the better and more cost-effective option. During the next several years a plethora of applications will emerge to take advantage of GPU computing. I am not exaggerating when I say that entire industries will be transformed by this technology.

In 2007, Nextbigfuture was covering the Tesla GPGPU