Deep learning

20% homework. 80% final project.

-Neural networks architectures: multilayer perceptron, convolutional neural networks (CNN), recurrent neural networks (RNN), long-short memory machines (LSTM), residual networks.

-The backpropagation algorithm

Stochastic gradient descent

-Loss functions and first-order methods for optimization (adaptive learning rate, Momentum, Nesterov, ADAM and more)

-Deep learning for classification and regression tasks

-From classification to object detection: the region CNN (RCNN) architecture and its various extensions, YOLO, SSD and other advanced techniques

-Semantic segmentation: the fully convolutional network (FCN) architecture for converting classification networks into segmentation ones and the mask-RCNN network for converting object detection networks to segmentation ones.

-Conditional random fields (CRF) and the usage with deep learning

-Metric learning - triplet loss, contractive loss, angular loss, Face recognition using deep learning-

- Deep learning for language modeling - a brief introduction to hidden markov models (HMM) and language models, n-grams, word embedding, the word-to-vec technique, machine translation, the encoder-decoder framework, the attention mechanism

- Deep learning for speech -  speech to text techniques, speaker identification, speech generation (the wavenet architecture)

Generative models - Autoencoders, Variational autoencoders, generative adversarial networks (GAN), the wasserstein GAN, cyclic GAN

- Deep learning for geometric data - spectral deep learning, pointNet.

Literature:

Book: Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, MIT press

Various recent research papers on advances in deep learning