Deep learning
20% homework. 80% final project.
-Neural networks architectures: multilayer perceptron, convolutional neural networks (CNN), recurrent neural networks (RNN), long-short memory machines (LSTM), residual networks.
-The backpropagation algorithm
Stochastic gradient descent
-Loss functions and first-order methods for optimization (adaptive learning rate, Momentum, Nesterov, ADAM and more)
-Deep learning for classification and regression tasks
-From classification to object detection: the region CNN (RCNN) architecture and its various extensions, YOLO, SSD and other advanced techniques
-Semantic segmentation: the fully convolutional network (FCN) architecture for converting classification networks into segmentation ones and the mask-RCNN network for converting object detection networks to segmentation ones.
-Conditional random fields (CRF) and the usage with deep learning
-Metric learning - triplet loss, contractive loss, angular loss, Face recognition using deep learning-
- Deep learning for language modeling - a brief introduction to hidden markov models (HMM) and language models, n-grams, word embedding, the word-to-vec technique, machine translation, the encoder-decoder framework, the attention mechanism
- Deep learning for speech - speech to text techniques, speaker identification, speech generation (the wavenet architecture)
Generative models - Autoencoders, Variational autoencoders, generative adversarial networks (GAN), the wasserstein GAN, cyclic GAN
- Deep learning for geometric data - spectral deep learning, pointNet.
Literature:
Book: Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, MIT press
Various recent research papers on advances in deep learning
