Monthly Archives: October 2017

Gamers Paid for Deep Learning and Self-Driving Cars Research

The gaming industry has a significant impact on deep learning and self-driving cars. GPUs fueled the progress in deep learning, as they enabled training much larger neural networks on large datasets.

For about 20 years, GPUs were mostly used by gamers. Every year NVIDIA was pouring billions of dollars into R&D to make better and faster GPUs to support better gaming experience. In the early 2000s, they started considering scientific computing and machine learning acceleration, or, more generally, general purpose GPU computation (GPGPU). They released the first version of CUDA, a parallel computing platform, in 2007. CUDA made it much easier to program for GPUs, which led to more experimentation by researchers.

In 2012, SuperVision group (Alex Krizhevsky, Geoffrey Hinton, and Ilya Sutskever) used CUDA to develop the AlexNet model that won the ImageNet Large Scale Visual Recognition Challenge. Perhaps, it was the event that brought the attention of machine learning community to the power of deep neural networks and GPU computation. Now NVIDIA dominates the market of hardware for training deep neural networks and is moving the space of hardware for inference. That’s interesting to think that gamers of 90′s and 2000′s paid for R&D of the deep learning hardware.

Continue reading