Deep Learning

From MyInfoRepo
Jump to navigation Jump to search

Placeholder, will be filled with content soon. Notes at: Notes:Deep Learning


...

...

Convolutional Neural Network (CNN) 3rd lecture

End-to-end learning in deep learning means there is no feature extraction. Kernels are used to filter images and detect features.

We look at an example network for an image. The network will be convoluted, a non-linearity function is applied, spatial pooling is applied and the image is combined for the next layer. By using multiple features in a convolution, we can detect different features. After convolution, we can apply a non-linearity to 'remove' all negative values from the data. By applying spatial pooling, a reduction of the image size, we create a lower resolution image. This is done to enhance effectiveness of then feature detection. We can look at larger features and gain a smaller model (activation maps do not have to be stored) to train on in the backpropagation step. By stacking images on top of each other after the spatial pooling, we keep the feature information for the next layer.

Toeplitz matrix (diagonal-constant matrix) is the same as convolution. Some thing to notice about these matrices: they are sparse, local and share parameters.

CNN is by design a limited parameter version of feed forward. This means we have less parameters and less flexibility. This is preferred, since we can learn faster and with convolution we target features that are expected, meaning we can distinguish easier.

Shifting and convolving is commutative (either can be done first, not changing the result).


...

...