lecture 3

Classification Networks [GoogleNet, AlexNet, ResNet], Detection Networks [SSD, YOLO, Fast/Faster, R-CNN], Segmentation Networks [FPN, FCN, SegNet]

Convolutional Neural Networks

Challenge1: Choosing the Right Deep Learning Network, Challenge2: Billions of multiply-accumulate operations and tens of megabytes of parameter data

<p>Caffe, TensorFlow, Mxnet</p> Signup and view all the answers

<p>Pruning is the process of eliminating redundant weights in neural networks while keeping the accuracy loss as low as possible. The benefits of pruning include reducing the model size, reducing runtime, and improving memory bandwidth utilization.</p> Signup and view all the answers

<p>Coarse-grained pruning involves setting the corresponding weights to 0 for an entire 3D kernel. Fine-grained pruning, on the other hand, cuts off redundant connections in the neural network by setting specific weights to 0.</p> Signup and view all the answers

<p>The three aspects involved in coarse-grained pruning are sparsity determination, channel selection, and sensitivity analysis.</p> Signup and view all the answers

<p>Iterative pruning is a process where the number of model parameters is gradually reduced while minimizing accuracy loss. Pruning results in accuracy loss, but retraining or fine-tuning can help recover accuracy. The reduction parameter is gradually increased in each iteration to aid in accuracy recovery during the fine-tuning stage.</p> Signup and view all the answers

<p>Quantization and channel pruning techniques are employed in neural networks to achieve high performance and energy efficiency while minimizing the degradation in accuracy. Quantization allows the use of integer computing units and representation of weights and activations with lower bits, reducing computing complexity. Channel pruning reduces overall required operations. These techniques enable faster speed, higher power efficiency, and memory bandwidth utilization in the inference process.</p> Signup and view all the answers

<p>The purpose of converting 32-bit floating-point weights and activations to 8-bit integer format in the AI quantizer is to reduce computing complexity without losing prediction accuracy. This conversion enables the use of fixed-point network models, which require less memory bandwidth and provide faster speed and higher power efficiency compared to floating-point models.</p> Signup and view all the answers

<p>The AI quantizer supports common layers in neural networks, including but not limited to convolution, pooling, fully connected, and batchnorm layers.</p> Signup and view all the answers

<p>The three approaches of deep learning are supervised learning, semi-supervised learning, and unsupervised learning.</p> Signup and view all the answers

<p>The types of neural networks used in deep learning are deep neural networks (DNN), convolutional neural networks (CNN), and recurrent neural networks (RNN) including long short-term memory (LSTM) and gated recurrent units (GRU).</p> Signup and view all the answers

<p>Supervised learning is based on labeled data sets and aims to approximate desired outputs, while unsupervised learning is based on no data labels and does not have specific desired outputs.</p> Signup and view all the answers