Knowledge Distillation, Hardware Optimization, Pruning

Reducing the precision of the parameters and activations in the network

Training a smaller network to mimic the behavior of a larger, pre-trained network (correct)

Removing unnecessary neurons and connections in the network

Using techniques such as singular value decomposition and low-rank factorization

Quantization

Model compression

Pruning

Using Graphical Processing Units (GPU) or Tensor Processing Units (TPU (correct)

To reduce the number of parameters and computations required during inference (correct)

To decrease memory usage and computation time

To train a smaller network to mimic the behavior of a larger, pre-trained network

To use techniques such as singular value decomposition and low-rank factorization