Deep Learning and Neural Networks [PDF]

Feb 6, 2014 - Advanced Research Seminar I/III ... This course provides an overview of Deep Learning and Neural Networks;

3 downloads 17 Views 40KB Size

Recommend Stories


An introduction to Neural Networks and Deep Learning
Almost everything will work again if you unplug it for a few minutes, including you. Anne Lamott

Hyphenation using deep neural networks
Come let us be friends for once. Let us make life easy on us. Let us be loved ones and lovers. The earth

Designing, Visualizing and Understanding Deep Neural Networks
Never wish them pain. That's not who you are. If they caused you pain, they must have pain inside. Wish

All-optical machine learning using diffractive deep neural networks
Everything in the universe is within you. Ask all from yourself. Rumi

Deep Learning of Graphs with Ngram Convolutional Neural Networks
At the end of your life, you will never regret not having passed one more test, not winning one more

learning hierarchical speech representations using deep convolutional neural networks
Ask yourself: What kind of legacy do you want to leave behind? Next

Effectiveness of Unsupervised Training in Deep Learning Neural Networks
I cannot do all the good that the world needs, but the world needs all the good that I can do. Jana

[PDF] Download Neural Networks
I want to sing like the birds sing, not worrying about who hears or what they think. Rumi

[PDF] Download Neural Networks
Sorrow prepares you for joy. It violently sweeps everything out of your house, so that new joy can find

Deep Neural Networks in Machine Translation
If you feel beautiful, then you are. Even if you don't, you still are. Terri Guillemets

Idea Transcript


Deep Learning and Neural Networks Advanced Research Seminar I/III Graduate School of Information Science Nara Institute of Science and Technology January 2014

Instructor: Kevin Duh, IS Building Room A-705 Office hours: after class, or appointment by email ([email protected] where x=kevinduh)

Course Description Deep Learning is a family of methods that exploits using deep architectures to learn high-level feature representations from data. Recently, these methods have helped researchers achieve impressive results in various fields within Artificial Intelligence, such as speech recognition, computer vision, and natural language processing. This course provides an overview of Deep Learning and Neural Networks; the goal is to establish a foundational understanding at a level sufficient for students to start reading research papers in this exciting and growing area. Prerequisites: basic calculus, probability, linear algebra.

Course Schedule Jan 14, 16, 21, 23 (9:20-10:50am) @ IS Building Room L2 Lecture 1 (Jan 14): Machine Learning background & Neural Networks Slides (pdf), Video (HD), Video (Youtube) Recommended reading: Chapter 5 of Bishop's book Pattern Recognition and Machine Learning Lecture 2 (Jan 16): Two Types of Deep Architectures: Deep Belief Nets (DBN) and Stacked Auto-Encoders (SAE) Slides (pdf), Video (HD), Video (Youtube) Recommended reading: Sections 1 & 4 of Bengio's monograph: Learning Deep Architectures for AI Lecture 3 (Jan 21): Applications in Computer Vision, Speech Recognition, and Language Modeling Slides (pdf), Video (HD), Video (Youtube) Recommended reading: [Hinton, et. al., IEEE SPM 2012], [Le, et. al., ICML2012], [Mikolov, et. al., Interspeech 2010] Lecture 4 (Jan 23): Advanced Topics in Optimization (Hessian-free optimization, Dropout, Large-scale distributed training, Hyper-parameter search) Slides (pdf), Video (HD), Video (Youtube) Recommended reading: [Martens, ICML2010], [Hinton, et. al., 2012], [Dean, et. al., NIPS2012], [Bergstra, et. al., NIPS2011] Two video options are available: [1] Video (HD) includes slide synchronization and requires Adobe Flash Player version 10 or above. [2] Video (Youtube) may be faster to load and is recommended if you have trouble with Video (HD). If you find errors, typos, or bugs in the slides/video, please let me know.

Useful References 1. Short surveys and tutorials: Yoshua Bengio's monograph (available online): Learning Deep Architectures for AI Yann LeCun & Marc'Aurelio Ranzato's ICML2013 tutorial (computer vision perspective) Richard Socher et. al.'s NAACL2013 tutorial (natural language processing perspective) Li Deng's talk at Johns Hopkins University CSLP (speech recognition perspective) 2. In-depth lectures and books: Hugo Larochelle's lecture videos and slides at U. Sherkbrooke Geoff Hinton's Coursera course Chris Bishop's book (worth buying!): Pattern Recognition and Machine Learning 3. To go even deeper: Play with the Python Theano code samples Check out the numerous up-to-date news and references on DeepLearning.net Last modified: Thu Feb 6 18:14:26 JST 2014

Smile Life

When life gives you a hundred reasons to cry, show life that you have a thousand reasons to smile

Get in touch

© Copyright 2015 - 2024 PDFFOX.COM - All rights reserved.