Monthly Archives: October 2017 What I Learned from Lessons 1-3 is a great deep learning course for those who prefer to learn by doing. Unlike other courses, here you will build a deep neural network that achieves good results in an image recognition problem in an hour. You start from working code and then dig deeper into the theory and the ways to improve your algorithm. Authors of the course are Jeremy Howard and Rachel Thomas.

I had prior knowledge of machine learning in general and neural networks in particular. I completed Andrew Ng’s Machine Learning course on Coursera, read a lot of articles, papers, and books, including parts of the Deep Learning book, and I’ve been building machine learning applications at work for more than a year. Nevertheless, I learned a lot from lessons. When you read books or papers, you learn a lot about the mathematics of deep learning, and little about how to use them in practice. is very helpful to learn practical aspects: how to split the dataset, how to choose good values of hyperparameters, how to prevent overfitting.

So far, I went through the first three lessons. Lesson 1 is mostly about setting up a development environment: set up an AWS account, create a virtual machine on Amazon EC2, run Jupyter Notebook there and execute the code that trains a convolutional neural network. The code sets up a Vgg16 CNN, loads weights pretrained on the ImageNet dataset, finetunes the model on a dataset from Dogs vs. Cats Kaggle competition, and then makes a prediction on test data to submit to Kaggle. I was doing something like that before, but it’s still surprising to me how easy it is to do transfer learning with CNN’s and get great results. Very inspiring.

They use Keras with Theano backend (you can use TensorFlow backend too, all the code still works).

Continue reading

Gamers Paid for Deep Learning and Self-Driving Cars Research

The gaming industry has a significant impact on deep learning and self-driving cars. GPUs fueled the progress in deep learning, as they enabled training much larger neural networks on large datasets.

For about 20 years, GPUs were mostly used by gamers. Every year NVIDIA was pouring billions of dollars into R&D to make better and faster GPUs to support better gaming experience. In the early 2000s, they started considering scientific computing and machine learning acceleration, or, more generally, general purpose GPU computation (GPGPU). They released the first version of CUDA, a parallel computing platform, in 2007. CUDA made it much easier to program for GPUs, which led to more experimentation by researchers.

In 2012, SuperVision group (Alex Krizhevsky, Geoffrey Hinton, and Ilya Sutskever) used CUDA to develop the AlexNet model that won the ImageNet Large Scale Visual Recognition Challenge. Perhaps, it was the event that brought the attention of machine learning community to the power of deep neural networks and GPU computation. Now NVIDIA dominates the market of hardware for training deep neural networks and is moving the space of hardware for inference. That’s interesting to think that gamers of 90′s and 2000′s paid for R&D of the deep learning hardware.

Continue reading