1. Unsupervised
1.1. Restricted Boltzmann Machines
1.1.1. A fast learning algorithm for deep belief nets
1.2. Discriminative RBMS
1.2.1. Classification using Discriminative Restricted Boltzmann Machines
1.3. Conditional RBMS
1.3.1. Modeling Human Motion Using Binary Latent Variables
1.4. Hybrid RBMS
1.5. Autoencoders
1.5.1. Cat paper
1.6. Semi-supervised Learning with Deep Generative Models
1.7. Mean covariance RBM
1.8. Spike and slab RBM
1.9. DRAW: A Recurrent Neural Network For Image Generation
1.9.1. Implementation in Theano
1.10. An Infinite Restricted Boltzmann Machine
2. Feed-forward networks
2.1. Backpropagation
2.1.1. Rprop
2.1.1.1. Climin implementation
2.1.2. RMSProp
2.1.2.1. Climin implementation
2.1.3. Feedback alignment
2.1.4. Efficient backprop
2.1.5. Fundamental deep learning problem
2.1.6. AdaDelta
2.1.6.1. Climin implementation
2.1.6.2. In Caffe
2.1.7. Adam
2.1.7.1. Implementation by author
2.1.8. AdaGrad
2.1.9. Conjugate gradient
2.1.9.1. Climin implementation
2.1.10. Quasi-Newton
2.1.10.1. Climin implementation
2.1.11. Gradient Descent
2.1.11.1. Climin implementation
2.1.12. Automatic differentiation in machine learning: a survey
2.2. Linear rectifier
2.2.1. Rectified Linear Units Improve Restricted Boltzmann Machines
2.3. Sparsity
2.3.1. Dropout
2.3.2. DropConnect
2.3.3. Weight decay
2.4. Hessian-free optimization
2.5. Nesterov momentum
2.5.1. On the importance of initialization and momentum in deep learning
2.5.2. Simplified Nesterov momentum in section 3.5
3. Recurrent neural networks
3.1. Advances in optimizing recurrent networks
3.2. Long-Short Term Memory
3.3. Reservoir computing
3.3.1. Echo State Networks
3.3.2. Liquid state machines
3.3.3. SORN: a self-organizing recurrent neural network
3.4. Generating Text with Recurrent Neural Networks
3.5. Neural Turing Machines
3.6. DRAW: A Recurrent Neural Network For Image Generation
3.7. Learning to Execute
4. Convolutional neural networks
4.1. deeplearning.net tutorial
4.2. Feature extraction using convolution
4.3. Pooling
4.4. CIFAR-10
4.5. ImageNet Classification with Deep Convolutional Neural Networks
4.6. DeepFace
4.7. Visualizing and Understanding Convolutional Networks
4.8. FaceNet: A Unified Embedding for Face Recognition and Clustering
5. Learning
5.1. UFLDL Tutorial
5.2. Deep Learning in Neural Networks: An Overview
5.3. UCLA Summer School
5.4. Deep Learning tutorial by Ruslan Salakhutdinov
5.5. Deep Learning book by Yoshua Bengio et al
5.6. VGG Convolutional Neural Networks Practical
5.7. CS231n: Convolutional Neural Networks for Visual Recognition
5.8. CS224d: Deep Learning for Natural Language Processing
5.9. comp.ai.neura-networks
5.10. Supervised Sequence Labelling with Recurrent Neural Networks
6. Future
6.1. Deep Learning of Representations: Looking Forward
6.2. Scaling Up Deep Learning
7. Bayesian networks
7.1. Hierarchical temporal memory
7.1.1. Numenta whitepaper
7.1.2. Hierarchical Temp oral Memory Cortical Learning Algorithm for Pattern Recognition on Multi-core Architectures
7.1.3. Towards a Mathematical Theory of Cortical Micro-circuits
7.2. Learning with Hierarchical-Deep Models
7.3. Theory-based Bayesian models of inductive learning and reasoning
8. Biology
8.1. Towards Biologically Plausible Deep Learning
9. Pretrained models
9.1. MatConvNet
9.2. VGG: Very Deep Convolutional Networks for Large-Scale Visual Recognition
9.3. VGG: Return of the Devil in the Details: Delving Deep into Convolutional Networks
9.4. Caffe reference models
9.5. Caffe community models
10. Transfer learning
10.1. CNN Features off-the-shelf: an Astounding Baseline for Recognition
11. Practical
11.1. Practical Recommendations for Gradient-Based Training of Deep Architectures
11.2. Efficient BackProp
11.3. A Practical Guide to Training Restricted Boltzmann Machines
11.4. A Brief Overview of Deep Learning (Ilya Sutskever)
11.5. Stochastic Gradient Descent Tricks
12. Berkeley Semantic Boundaries Dataset and Benchmark (SBD)
13. Berkeley Video Segmentation Dataset (BVSD)
13.1. train
13.2. test
14. Berkeley Segmentation Data Set 300 (BSDS300)
15. Software
15.1. C/C++
15.1.1. RNNLM
15.1.2. GPUMLib
15.1.3. Shark
15.1.4. libccv
15.1.5. RNNLIB
15.1.6. OverFeat
15.1.7. ClConvolve (OpenCL)
15.1.8. cxxnet
15.1.9. EBLearn
15.1.10. currennt
15.1.11. nnForge
15.2. Java
15.2.1. Deeplearning4j
15.2.2. Encog
15.2.3. Nd4j
15.3. Javascript
15.3.1. ConvNetJS
15.4. Lua
15.4.1. Torch7
15.4.1.1. Neural Turing Machine
15.4.1.2. dp package
15.4.1.3. rnn: recurrent neural networks
15.4.1.4. Resources (Cheatsheet)
15.4.1.5. LSTM Units
15.4.1.6. Autograd
15.5. Matlab
15.5.1. DeepLearningToolbox
15.5.2. Matlab neural network toolbox
15.5.2.1. Documentation
15.5.3. convolutionalRBM
15.5.4. RBMS
15.5.4.1. Medal
15.5.4.2. Deepmat
15.5.4.3. Ruslan Salakhutdinov's examples (DBN, RBM)
15.5.5. convolutional nets
15.5.5.1. MatConvNet
15.5.5.2. cudacnn
15.5.5.3. myCNN
15.5.5.4. ConvNet
15.6. Python
15.6.1. Caffe
15.6.1.1. NVIDIA DIGITS
15.6.1.2. Expresso
15.6.1.3. R-CNN (object detector)
15.6.1.4. DeepDetect
15.6.1.5. NLPCaffe
15.6.2. climin
15.6.3. cudamat
15.6.4. cuda-convnet
15.6.5. cuda-convnet2
15.6.6. deepnet
15.6.7. PyBrain
15.6.8. Theano
15.6.8.1. Start here
15.6.8.2. PyLearn2
15.6.8.3. Blocks
15.6.8.4. Lasagne
15.6.8.4.1. recurrent layers
15.6.8.5. PDNN
15.6.8.6. Theanets
15.6.8.7. Keras
15.6.9. nolearn
15.6.10. CUV (RBMs)
15.6.11. GroundHog (RNNs)
15.6.12. Decaf
15.6.13. NervanaGPU
15.6.14. neon
15.6.15. Passage
15.6.16. List of Python toolkits
15.6.17. Chainer
15.6.18. Computation Graph Toolkit (CGT)
15.6.19. Gensim
15.6.20. Hebel
15.6.21. Brainstorm
15.6.22. Autograd
15.7. Service
15.7.1. Ersatzlabbs
15.8. Lists
15.8.1. Comparison of convnet implementations
15.8.2. deeplearning.net software links
15.8.3. Toronto deep learning codes
15.8.4. Teglor
15.9. iOS
15.9.1. DeepBeliefSDK
15.10. Julia
15.10.1. KUnet.jl
15.10.1.1. Beginning deep learning with 500 lines of Julia
15.10.2. Mocha.jl
15.10.3. Boltzmann.jl
15.11. Other
15.11.1. Ayda
16. Places
16.1. Universities
16.1.1. University of Toronto
16.1.1.1. Geoffrey Hinton
16.1.1.2. Ilya Sutskever
16.1.1.3. Alex Krizhevsky
16.1.1.4. Ruslan Salakhutdinov
16.1.1.5. Volodymyr Vmnih
16.1.1.6. Alex Graves
16.1.1.7. Nitish Srivastava
16.1.2. New York University
16.1.2.1. Yann LeCun
16.1.3. University of Montreal
16.1.3.1. Yoshua Bengio
16.1.3.2. Tomas Mikolov
16.1.4. University of Standford
16.1.4.1. Andrew Ng
16.1.4.2. Andrej Karpathy
16.1.5. Computer Vision and Active Perception Lab, KTH Royal Institute of Technology
16.1.5.1. Ali Sharif Razavian
16.1.5.2. Hossein Azizpour
16.1.5.3. Josephine Sullivan
16.2. Companies
16.2.1. Big names
16.2.1.1. Google
16.2.1.1.1. DeepMind
16.2.1.2. Facebook
16.2.1.3. Microsoft Research
16.2.1.4. Baidu
16.2.1.5. Flickr
16.2.1.5.1. Park or bird
16.2.2. Startups
16.2.2.1. Numenta
16.2.2.1.1. Jeff Hawkins
16.2.2.1.2. Dileep George
16.2.2.2. Vicarious
16.2.2.3. Metamind
16.2.2.3.1. Richard Socher
16.2.2.4. Clarify
16.2.2.4.1. Matthew D. Zeiler
16.2.2.5. Enlitic
16.2.2.5.1. Jeremy Howard
16.2.2.6. Nervana
16.2.2.7. Whetlab
16.2.2.7.1. Ryan Adams
16.2.2.7.2. Hugo Larochelle
16.2.2.7.3. Jasper Snoek
16.2.2.7.4. Kevin Swersky
17. Reinforcement learning
17.1. Q-learning tutorial
17.2. Playing Atari with Deep Reinforcement Learning
17.2.1. Nathan's implementation
17.2.2. Brian's implementation
17.2.3. Our implementation
17.2.4. Deep Q Learning Demo
17.3. Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method
17.4. A Monte-Carlo AIXI Approximation
17.5. Human-level control through deep reinforcement learning
17.6. Tutorial on Multi-Agent Reinforcement Learning
17.7. Distributed Deep Q-Learning
17.7.1. Code
18. Multimodal networks
18.1. Ruslan Salakhutdinov's video
18.2. Multimodal Learning with Deep Boltzmann Machines
19. Datasets
19.1. Images
19.1.1. MNIST
19.1.2. CIFAR-10/100
19.1.3. ImageNet
19.1.4. Caltech101
19.1.5. Caltech256
19.1.6. Berkeley Segmentation Data Set 500 (BSDS500)
19.2. Image-Text
19.2.1. The generalized 1M image-caption corpus
19.2.2. Microsoft COCO
19.3. Faces
19.3.1. Labeled Faces in the Wild
19.3.2. Weakly Labeled Face Databases
19.3.3. YouTube faces