Is my neural network too big to fit into GPU?
It is well known that GPUs can significantly accelerate neural network training. However, there are lots of questions around the use of GPUs, especially for beginners. In this talk, we will dissect a particular convolutional Neural Network (NN) and use it as an example to answer these frequently asked questions. We will illustrate how to summarize and visualize the architecture of a NN, from which we will make a coarse estimate of memory requirement. Then, we’ll show how to accurately check the GPU memory usage at runtime and provide several advice in the case it runs out of GPU memory. The live demo in the talk uses Keras interface on Graham cluster and the source code will be provided after the talk. _________________________________________________ This webinar was presented by Weiguang Guan (SHARCNET) on September 23rd, 2020, as a part of a series of regular biweekly General Interest webinars ran by SHARCNET. The webinars cover different high performance computing (HPC) topics, are approximately 45 minutes in length, and are delivered by experts in the relevant fields. Further details can be found on this web page: https://www.sharcnet.ca/help/index.php/Online_Seminars . Subscribe to our twitter account (@SHARCNET) to stay updated about our upcoming webinars. SHARCNET is a consortium of 19 Canadian academic institutions who share a network of high performance computers (http://www.sharcnet.ca). SHARCNET is a part of Compute Ontario (http://computeontario.ca/) and Compute Canada (https://computecanada.ca).
It is well known that GPUs can significantly accelerate neural network training. However, there are lots of questions around the use of GPUs, especially for beginners. In this talk, we will dissect a particular convolutional Neural Network (NN) and use it as an example to answer these frequently asked questions. We will illustrate how to summarize and visualize the architecture of a NN, from which we will make a coarse estimate of memory requirement. Then, we’ll show how to accurately check the GPU memory usage at runtime and provide several advice in the case it runs out of GPU memory. The live demo in the talk uses Keras interface on Graham cluster and the source code will be provided after the talk. _________________________________________________ This webinar was presented by Weiguang Guan (SHARCNET) on September 23rd, 2020, as a part of a series of regular biweekly General Interest webinars ran by SHARCNET. The webinars cover different high performance computing (HPC) topics, are approximately 45 minutes in length, and are delivered by experts in the relevant fields. Further details can be found on this web page: https://www.sharcnet.ca/help/index.php/Online_Seminars . Subscribe to our twitter account (@SHARCNET) to stay updated about our upcoming webinars. SHARCNET is a consortium of 19 Canadian academic institutions who share a network of high performance computers (http://www.sharcnet.ca). SHARCNET is a part of Compute Ontario (http://computeontario.ca/) and Compute Canada (https://computecanada.ca).