Distinguished Seminar Series on Data Science & Artificial Intelligence - "Conditional Computation for Efficient DCNN-Based Inference"

Conditional Computation for Efficient DCNN-Based Inference

Deep neural networks (DNNs) have been successfully applied to solve many problems. If only accuracy is considered, DNNs can even perform better than humans for tasks such as image classification and object detection. However, the inference cost of most DNN models is usually high and not conducive to real-time processing. This is because all neurons in the network must be calculated to make each single decision. In order to reduce the cost of inference, we can use knowledge distillation or model compression to design small neural networks. However, the small networks so obtained may be too simple to solve the problem well. Another method is "conditional computing", that is, selecting a small part of the network to make decisions based on certain conditions. This approach is more like the human "brain" because even though the entire network may be large, for each input we only use a small part of the system to calculate the corresponding output. This talk will first review several existing neural network models based on conditional computing, and then introduce the model recently proposed by our research team. Before listening to the report, you may try to think about the following questions: 1) What conditions should we use to select the "part" of the network to make decisions? 2) Which "parts" of the network can we use to make decisions? 3) How should we design the network to make conditional computing possible? And so on.