Idea Transcript
Navigation
Start Here Blog Books About Contact Search...
Need help with Deep Learning? Take the FREE Mini-Course
What is Deep Learning? by Jason Brownlee on August 16, 2016 in Deep Learning
Deep Learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks. If you are just starting out in the field of deep learning or you had some experience with neural networks some time ago, you may be confused. I know I was confused initially and so were many of my colleagues and friends who learned and used neural networks in the 1990s and early 2000s. The leaders and experts in the field have ideas of what deep learning is and these specific and nuanced perspectives shed a lot of light on what deep learning is all about. In this post, you will discover exactly what deep learning is by hearing from a range of experts and leaders in the field. Let’s dive in.
What is Deep Learning? Photo by Kiran Foster, some rights reserved.
Deep Learning is Large Neural Networks Andrew Ng from Coursera and Chief Scientist at Baidu Research formally founded Google Brain that eventually resulted in the productization of deep learning technologies across a large number of Google services. He has spoken and written a lot about what deep learning is and is a good place to start. In early talks on deep learning, Andrew described deep learning in the context of traditional artificial neural networks. In the 2013 talk titled “Deep Learning, Self-Taught Learning and Unsupervised Feature Learning” he described the idea of deep learning as: Using brain simulations, hope to: – Make learning algorithms much better and easier to use. – Make revolutionary advances in machine learning and AI. I believe this is our best shot at progress towards real AI Later his comments became more nuanced. The core of deep learning according to Andrew is that we now have fast enough computers and enough data to actually train large neural networks. When discussing why now is the time that deep learning is taking off at ExtractConf 2015 in a talk titled “What data scientists should know about deep learning“, he commented: very large neural networks we can now have and … huge amounts of data that we have access to He also commented on the important point that it is all about scale. That as we construct larger neural networks and train them with more and more data, their performance continues to increase. This is generally different to other machine learning techniques that reach a plateau in performance. for most flavors of the old generations of learning algorithms … performance will plateau. … deep learning … is the first class of algorithms … that is scalable. … performance just keeps getting better as you feed them more data He provides a nice cartoon of this in his slides:
Why Deep Learning? Slide by Andrew Ng, all rights reserved.
Finally, he is clear to point out that the benefits from deep learning that we are seeing in practice come from supervised learning. From the 2015 ExtractConf talk, he commented: almost all the value today of deep learning is through supervised learning or learning from labeled data Earlier at a talk to Stanford University titled “Deep Learning” in 2014 he made a similar comment: one reason that deep learning has taken off like crazy is because it is fantastic at supervised learning Andrew often mentions that we should and will see more benefits coming from the unsupervised side of the tracks as the field matures to deal with the abundance of unlabeled data available. Jeff Dean is a Wizard and Google Senior Fellow in the Systems and Infrastructure Group at Google and has been involved and perhaps partially responsible for the scaling and adoption of deep learning within Google. Jeff was involved in the Google Brain project and the development of large-scale deep learning software DistBelief and later TensorFlow. In a 2016 talk titled “Deep Learning for Building Intelligent Computer Systems” he made a comment in the similar vein, that deep learning is really all about large neural networks. When you hear the term deep learning, just think of a large deep neural net. Deep refers to the number of layers typically and so this kind of the popular term that’s been adopted in the press. I think of them as deep neural networks generally. He has given this talk a few times, and in a modified set of slides for the same talk, he highlights the scalability of neural networks indicating that results get better with more data and larger models, that in turn require more computation to train.
Results Get Better With More Data, Larger Models, More Compute Slide by Jeff Dean, All Rights Reserved.
Deep Learning is Hierarchical Feature Learning In addition to scalability, another often cited benefit of deep learning models is their ability to perform automatic feature extraction from raw data, also called feature learning. Yoshua Bengio is another leader in deep learning although began with a strong interest in the automatic feature learning that large neural networks are capable of achieving. He describes deep learning in terms of the algorithms ability to discover and learn good representations using feature learning. In his 2012 paper titled “Deep Learning of Representations for Unsupervised and Transfer Learning” he commented: Deep learning algorithms seek to exploit the unknown structure in the input distribution in order to discover good representations, often at multiple levels, with higher-level learned features defined in terms of lower-level features An elaborated perspective of deep learning along these lines is provided in his 2009 technical report titled “Learning deep architectures for AI” where he emphasizes the importance the hierarchy in feature learning. Deep learning methods aim at learning feature hierarchies with features from higher levels of the hierarchy formed by the composition of lower level features. Automatically learning features at multiple levels of abstraction allow a system to learn complex functions mapping the input to the output directly from data, without depending completely on human-crafted features. In the soon to be published book titled “Deep Learning” co-authored with Ian Goodfellow and Aaron Courville, they define deep learning in terms of the depth of the architecture of the models. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones. If we draw a graph showing how these concepts are built on top of each other, the graph is deep, with many layers. For this reason, we call this approach to AI deep learning. This is an important book and will likely become the definitive resource for the field for some time. The book goes on to describe multilayer perceptrons as an algorithm used in the field of deep learning, giving the idea that deep learning has subsumed artificial neural networks. The quintessential example of a deep learning model is the feedforward deep network or multilayer perceptron (MLP). Peter Norvig is the Director of Research at Google and famous for his textbook on AI titled “Artificial Intelligence: A Modern Approach“. In a 2016 talk he gave titled “Deep Learning and Understandability versus Software Engineering and Verification” he defined deep learning in a very similar way to Yoshua, focusing on the power of abstraction permitted by using a deeper network structure. a kind of learning where the representation you form have several levels of abstraction, rather than a direct input to output
Why Call it “Deep Learning”? Why Not Just “Artificial Neural Networks”? Geoffrey Hinton is a pioneer in the field of artificial neural networks and co-published the first paper on the backpropagation algorithm for training multilayer perceptron networks. He may have started the introduction of the phrasing “deep” to describe the development of large artificial neural networks. He co-authored a paper in 2006 titled “A Fast Learning Algorithm for Deep Belief Nets” in which they describe an approach to training “deep” (as in a many layered network) of restricted Boltzmann machines. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. This paper and the related paper Geoff co-authored titled “Deep Boltzmann Machines” on an undirected deep network were well received by the community (now cited many hundreds of times) because they were successful examples of greedy layer-wise training of networks, allowing many more layers in feedforward networks. In a co-authored article in Science titled “Reducing the Dimensionality of Data with Neural Networks” they stuck with the same description of “deep” to describe their approach to developing networks with many more layers than was previously typical. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data. In the same article, they make an interesting comment that meshes with Andrew Ng’s comment about the recent increase in compute power and access to large datasets that has unleashed the untapped capability of neural networks when used at larger scale. It has been obvious since the 1980s that backpropagation through deep autoencoders would be very effective for nonlinear dimensionality reduction, provided that computers were fast enough, data sets were big enough, and the initial weights were close enough to a good solution. All three conditions are now satisfied. In a talk to the Royal Society in 2016 titled “Deep Learning“, Geoff commented that Deep Belief Networks were the start of deep learning in 2006 and that the first successful application of this new wave of deep learning was to speech recognition in 2009 titled “Acoustic Modeling using Deep Belief Networks“, achieving state of the art results. It was the results that made the speech recognition and the neural network communities take notice, the use “deep” as a differentiator on previous neural network techniques that probably resulted in the name change. The descriptions of deep learning in the Royal Society talk are very backpropagation centric as you would expect. Interesting, he gives 4 reasons why backpropagation (read “deep learning”) did not take off last time around in the 1990s. The first two points match comments by Andrew Ng above about datasets being too small and computers being too slow.
What Was Actually Wrong With Backpropagation in 1986? Slide by Geoff Hinton, all rights reserved.
Deep Learning as Scalable Learning Across Domains Deep learning excels on problem domains where the inputs (and even output) are analog. Meaning, they are not a few quantities in a tabular format but instead are images of pixel data, documents of text data or files of audio data. Yann LeCun is the director of Facebook Research and is the father of the network architecture that excels at object recognition in image data called the Convolutional Neural Network (CNN). This technique is seeing great success because like multilayer perceptron feedforward neural networks, the technique scales with data and model size and can be trained with backpropagation. This biases his definition of deep learning as the development of very large CNNs, which have had great success on object recognition in photographs. In a 2016 talk at Lawrence Livermore National Laboratory titled “Accelerating Understanding: Deep Learning, Intelligent Applications, and GPUs” he described deep learning generally as learning hierarchical representations and defines it as a scalable approach to building object recognition systems: deep learning [is] … a pipeline of modules all of which are trainable. … deep because [has] multiple stages in the process of recognizing an object and all of those stages are part of the training”
Deep Learning = Learning Hierarchical Representations Slide by Yann LeCun, all rights reserved.
Jurgen Schmidhuber is the father of another popular algorithm that like MLPs and CNNs also scales with model size and dataset size and can be trained with backpropagation, but is instead tailored to learning sequence data, called the Long Short-Term Memory Network (LSTM), a type of recurrent neural network. We do see some confusion in the phrasing of the field as “deep learning”. In his 2014 paper titled “Deep Learning in Neural Networks: An Overview” he does comment on the problematic naming of the field and the differentiation of deep from shallow learning. He also interestingly describes depth in terms of the complexity of the problem rather than the model used to solve the problem. At which problem depth does Shallow Learning end, and Deep Learning begin? Discussions with DL experts have not yet yielded a conclusive response to this question. […], let me just define for the purposes of this overview: problems of depth > 10 require Very Deep Learning. Demis Hassabis is the founder of DeepMind, later acquired by Google. DeepMind made the breakthrough of combining deep learning techniques with reinforcement learning to handle complex learning problems like game playing, famously demonstrated in playing Atari games and the game Go with Alpha Go. In keeping with the naming, they called their new technique a Deep Q-Network, combining Deep Learning with Q-Learning. They also name the broader field of study “Deep Reinforcement Learning”. In their 2015 nature paper titled “Human-level control through deep reinforcement learning” they comment on the important role of deep neural networks in their breakthrough and highlight the need for hierarchical abstraction. To achieve this,we developed a novel agent, a deep Q-network (DQN), which is able to combine reinforcement learning with a class of artificial neural network known as deep neural networks. Notably, recent advances in deep neural networks, in which several layers of nodes are used to build up progressively more abstract representations of the data, have made it possible for artificial neural networks to learn concepts such as object categories directly from raw sensory data. Finally, in what may be considered a defining paper in the field, Yann LeCun, Yoshua Bengio and Geoffrey Hinton published a paper in Nature titled simply “Deep Learning“. In it, they open with a clean definition of deep learning highlighting the multi-layered approach. Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. Later the multi-layered approach is described in terms of representation learning and abstraction. Deep-learning methods are representation-learning methods with multiple levels of representation, obtained by composing simple but non-linear modules that each transform the representation at one level (starting with the raw input) into a representation at a higher, slightly more abstract level. […] The key aspect of deep learning is that these layers of features are not designed by human engineers: they are learned from data using a general-purpose learning procedure. This is a nice and generic a description, and could easily describe most artificial neural network algorithms. It is also a good note to end on.
Summary In this post you discovered that deep learning is just very big neural networks on a lot more data, requiring bigger computers. Although early approaches published by Hinton and collaborators focus on greedy layerwise training and unsupervised methods like autoencoders, modern state-of-the-art deep learning is focused on training deep (many layered) neural network models using the backpropagation algorithm. The most popular techniques are: Multilayer Perceptron Networks. Convolutional Neural Networks. Long Short-Term Memory Recurrent Neural Networks. I hope this has cleared up what deep learning is and how leading definitions fit together under the one umbrella. If you have any questions about deep learning or about this post, ask your questions in the comments below and I will do my best to answer them.
Frustrated With Your Progress In Deep Learning? What If You Could Develop A Network in Minutes …with just a few lines of Python Discover how in my new Ebook: Deep Learning With Python It covers self-study tutorials and end-to-end projects on topics like: Multilayer Perceptrons, Convolutional Nets and Recurrent Neural Nets, and more…
Finally Bring Deep Learning To Your Own Projects Skip the Academics. Just Results. Click to learn more.
About Jason Brownlee Jason Brownlee, Ph.D. is a machine learning specialist who teaches developers how to get results with modern machine learning methods via hands-on tutorials. View all posts by Jason Brownlee Õ
Weka Machine Learning Mini-Course
A Gentle Introduction to XGBoost for Applied Machine Learning
102 Responses to What is Deep Learning? REPLY
Gibachan August 16, 2016 at 7:24 am # If the deep learning is such great algorithm, do you think that other older algorithms (like SVM) are no longer efficient to solve our problems?
REPLY
Jason Brownlee August 16, 2016 at 8:59 am #
I think that SVM and similar techniques still have their place. It seems that the niche for deep learning techniques is when you are working with raw analog data, like audio and image data.
REPLY
Priyankar September 5, 2017 at 3:45 pm # Could you please give me some idea, how deep learning can be applied on social media data i.e. twitter.
REPLY
Jason Brownlee September 7, 2017 at 12:42 pm # Start by defining your problem: http://machinelearningmastery.com/how-to-define-your-machine-learning-problem/
REPLY
Tooba February 17, 2017 at 2:43 am #
first of all I would like to appreciate your effort. This is one of the best blog on deep learning I have read so far. Well I would like to ask you if we need to extract some data like advertising boards from image, what you suggest is better SVM or CNN or do you have any better algorithm than these two in your mind?
REPLY
Swapnil Pote March 17, 2017 at 1:22 am # CNN will give better result as compare to svm in image classification
REPLY
Ibrahim Amer September 6, 2017 at 10:11 pm #
CNN would be extremely better than SVM if and only if you have enough data. The reason that CNN would be better is that CNN work as an automatic feature extractor and you won’t need to bother yourself of feature selection and wondering if the extracted feature would weather work with the model or not. CNN extracts all possible features, from low-level features like edges to higher-level features like faces and objects.
REPLY
Alan Beckles MD MS August 16, 2016 at 11:12 am # Can CNNs perform tasks such as Medical Diagnosis or should they be combined with another technique such as Reinforement Learning to optimize performance?
REPLY
Jason Brownlee August 16, 2016 at 11:20 am # Generally, CNNs are really good at working with image data.
Medical Diagnosis seems like a really broad domain. You may want to narrow your scope and clearly define and frame your problem before selecting specific algorithms.
REPLY
Alan Beckles MD MS August 16, 2016 at 12:03 pm #
ECG interpretation may be a good problem for CNNs in that they are images. Another project is the development of a Consultant in Cardiovascular Disease analogous to MYCIN, an Infectious Disease Consultatant developed by Shortliffe & Buchanan @ Stanford ~ 40 years ago which was Rule Based.
REPLY
napoleon Boakye September 9, 2016 at 1:24 am # So Jason, what is the next discovery after “deep learning”?
REPLY
Jason Brownlee September 9, 2016 at 7:22 am # No idea Napoleon. Deep learning has enough potential to keep us busy for a long while.
REPLY
napoleon Boakye September 12, 2016 at 8:13 am # Okay
REPLY
Francesco D'Amore September 14, 2016 at 11:04 pm # Good overview. Take a look at this: http://deeplearning4j.org/ It could be a good tool for DL?
REPLY
Jason Brownlee September 15, 2016 at 8:22 am # Thanks Francesco.
REPLY
Jason Wills October 4, 2016 at 10:10 pm # hello, may deep learning apply to use in the stock market ? What I mean : it doesn’t just only use to draw with old data diagram and use the old model but also write down how is the next day to give the number forecast ?
REPLY
Jason Brownlee October 5, 2016 at 8:28 am # Hi Jason, deep learning may apply to the stock market. I am not an expert in finance so I cannot give you expert advice. Try it and see. You may be interested in this post on time series forecasting with deep learning: http://machinelearningmastery.com/time-series-prediction-with-deep-learning-in-python-with-keras/
REPLY
Jason Wills October 5, 2016 at 2:16 pm #
Thank for your reply, I have read some your posts and I am very impressed with your work. About myself , I just start to find out what is this filed and you have many experiences about them. I hope if you have some experiences about the finance especially in stock market…pls help me some reference to learn it by myself or find the “Tribute”as you mentioned
REPLY
Jason Brownlee October 6, 2016 at 9:29 am # Thanks Jason, it’s great to have you here.
REPLY
maisie Badami October 15, 2016 at 4:47 pm # loved it , thanks for the overview , answered to a lot of my question
I am trying to find a topic for my Master-PHD proposal in Deep Learning in medical diagnosis and just wondering if there is any hot topic in this field at the moment ? and how can I learn more about this special field of Deep Learning
REPLY
Jason Brownlee October 17, 2016 at 10:18 am # I’m glad to hear it was useful Maisie. I would suggest talking to medical diagnosis people about big open problems where there is access to lots of data.
REPLY
neha rahman October 20, 2016 at 5:15 am # i am looking for M tech thesis in this topic…help me explore new areas….
REPLY
Jason Brownlee October 20, 2016 at 8:40 am # Hi neha, the best person to talk to about research topic ideas is your advisor. Best of luck.
REPLY
Abbey November 14, 2016 at 4:39 am # Hi Jason,
Thank you so much for your post. I am trying to solve an open problem with regards to embedded short text messages on the social media which are abbreviation, symbol and others. For instance, take bf can be interpret as boy friend or best friend. The input can be represent as character but how can someone encode this as input in neural network, so it can learn and output the target at the same time. Please help. Regards Abbey
REPLY
Jason Brownlee November 14, 2016 at 7:46 am # Very cool problem Abbey. I would suggest starting off by collecting a very high-quality dataset of messages and expected translation. I would then suggest encoding the words as integers and use a word embedding to project the integer vectors into a higher dimensional space. Let me know how you go.
REPLY
Sam Wilson January 5, 2017 at 4:14 am # Hi, thanks for the good overview. In your opinion, on what field CNN could be used in developing countries? Because there seems less raw data than developed countries, i couldn’t think of any use of CNN in developing countries.
REPLY
Jason Brownlee January 5, 2017 at 9:41 am # Sorry Sam, I don’t know. CNNs are state of the art on many problems that have spatial structure (or structure that can be made spatial). Anything with images is a great start, domains like text and time series are also interesting.
REPLY
Danie Truter March 21, 2017 at 1:10 am #
Hi… I am an average developer in a developing country and my opinion is “yes”… if you find a way to get all these “disconnected” data together than you can help on gathering these data to make it easier for developing countries not to make the same mistakes as developed countries… thus bringing the cost down on “becoming” a developed country without the cost… the “research” exist… the implementation is the problem…
REPLY
Muhammad Faisal February 26, 2017 at 12:42 am # Hello Jason,
a very well and nicely explained article for the beginners. I would like to ask one question, Please tell me any specific example in the area of computer vision, where shallow learning (Conventional Machine Learning) is much better than Deep Learning.
REPLY
Jason Brownlee February 26, 2017 at 5:30 am # Great question, I’m not sure off hand. Computer Vision is not really my area of expertise.
REPLY
priyanka yemul March 13, 2017 at 9:23 pm # This article is useful for learning deep learning .Nice article
REPLY
Jason Brownlee March 14, 2017 at 8:17 am # Thanks priyanka.
REPLY
Chris Jarvis March 14, 2017 at 8:36 am # Wonderful summary of Deep Learning – I am doing an undergraduate dissertation/thesis on applying Artificial Intelligence to solving Engineering problems.
REPLY
Jason Brownlee March 14, 2017 at 8:47 am # Thanks Chris! Good luck with your thesis.
REPLY
Danie Truter March 21, 2017 at 1:01 am # Hi… I am just an average normal developer, but I find this article very informative… May I please ask one question:
If the “internet” and “line speed” was fast enough, would it mean these algorithms could learn itself or are the “programs” currently limited to human interaction during the learning stage… So my actual question: the “data” according to me is available -> “internet” BUT do we (humanity currently) already have the computational ability to make “sense” of the data via these algorithms AND are the software developed in such a way to ignore human approval?
REPLY
Jason Brownlee March 21, 2017 at 8:42 am #
The data needed to learn for a given problem varies from problem to problem. As does the source of data and the transmission of data from the source to the learning algorithm.
REPLY
Cyriac Peter March 22, 2017 at 5:56 am #
Dr Jason, this is an immensely helpful compilation. I researched quite a bit today to understand what Deep Learning actually is. I must say all articles were helpful, but yours make me feel satisfied about my research today. Thanks again. Based on my readings so far, I feel predictive analytics is at the core of both machine learning and deep learning is an approach for predictive analytics with accuracy that scales with more data and training. Would like to hear your thoughts on this.
REPLY
Jason Brownlee March 22, 2017 at 8:10 am # Thanks. I’m very glad to hear it. Predictive modeling is a sub-field of machine learning, and is by far the most useful area/the area of interest right now: http://machinelearningmastery.com/gentle-introduction-to-predictive-modeling/
REPLY
Tran Anh Tuan March 30, 2017 at 6:23 pm # This article is very interesting and useful for a beginner in machine learning like me.
I am thinking about a project (just for my hobby) of designing a stabilization controller for a DIY Quadrotor. Do you have any advice on how and where I should start off? Can algorithms like SVM be used in this specific purpose? Is micro controller (like Arduino) able to handle this problem? Thank you in advance
REPLY
Jason Brownlee March 31, 2017 at 5:52 am # Sounds like a lot of fun. Consider this process to work through your problem: http://machinelearningmastery.com/start-here/#process
REPLY
Murali April 19, 2017 at 8:39 pm # hi Is the Deep Learning is suitable for prediction of any diseases like Diabetes using data mining algorithms? If yes give some ideas to work in it
REPLY
Jason Brownlee April 20, 2017 at 9:24 am # It may be good, but try a suite of algorithms to see what works best on your problem. See this post: http://machinelearningmastery.com/a-data-driven-approach-to-machine-learning/
REPLY
arif May 7, 2017 at 6:29 pm # hi jason are you fine i read your article that help me out and the comment section also may i know that which is the latest algorithm in deep neural network i need your help about neural network . i am working on neural network thanks
REPLY
Jason Brownlee May 8, 2017 at 7:43 am # There are many deep learning algorithms. The most popular are MLPs for tabular data, CNNs for image data and LSTMs for sequence data.
REPLY
arif May 8, 2017 at 3:09 pm # thanks jason in research base which algorithm you suggest for me to work
REPLY
Jason Brownlee May 9, 2017 at 7:38 am # I would suggest you pick an area that most excites you.
REPLY
arif May 12, 2017 at 1:25 am # JASON I WANT TO WORK IN MEDICAL AREA OR IMPLEMENT IN TO MEDICAL SITE
REPLY
Jason Brownlee May 12, 2017 at 7:43 am # Great! I wish you the best of luck.
REPLY
shivam tripathi May 15, 2017 at 3:44 am # does deep learning is a solution of over-fitting problem in machine learning?
REPLY
Jason Brownlee May 15, 2017 at 5:54 am # No, deep learning methods can overfit like any other.
REPLY
Mani May 19, 2017 at 4:32 pm # Thanks for the great article. What is the best approach for classifying products based on product description?
REPLY
Jason Brownlee May 20, 2017 at 5:35 am # Perhaps use an LSTM or CNN or both combined.
REPLY
Tofa May 26, 2017 at 6:30 pm #
Lots of unnecessary points your explained which make difficult to understand what is actually deep learning is, also unnecessary explanaiton meke me bouring to read the document.
REPLY
Jason Brownlee June 2, 2017 at 11:53 am # Sorry to hear that.
REPLY
Anthony June 6, 2017 at 7:19 pm # Jason, What do you think is the future of deep learning? How many years do you think will it take before a new algorithm becomes popular?
REPLY
Jason Brownlee June 7, 2017 at 7:12 am # Hi Anthony, There is not one algorithm, but a population with the headliners: MLP, LSTM and CNN. I do not know where we are headed, sorry.
REPLY
Dubem June 16, 2017 at 10:02 pm # I am a student of computer science and am to present a seminar on deep learning, I av no idea of what is all about…..can I get articles dat can aid Me
REPLY
Jason Brownlee June 17, 2017 at 7:28 am # This post would be a good start. I do not have much general material, but if you want to know how to use deep learning to solve problems, start here: http://machinelearningmastery.com/start-here/#deeplearning
REPLY
Neel June 26, 2017 at 3:00 pm #
Hi Jason, I have been referring to a few of your blogs for my Machine Learning stuff. One striking feature of your blogs is simplicity which draws me regularly to this place! This is very helpful.:) Talking about Deep Learning vs traditional ML, the general conception is that Deep Learning beats a human being at its ability to do feature abstraction. Is this true? Also, could you tell me why Deep Learning fails to achieve more than many of the traditional ML algorithms for different datasets despite the assumed superiority of DL in feature abstraction over other algorithms?
REPLY
Jason Brownlee June 27, 2017 at 8:27 am # Deep learning is great at feature extraction and in turn state of the art prediction on what I call “analog data”, e.g. images, text, audio, etc. It can be used on tabular data (e.g. spreadsheet of numbers) but this is not it’s sweet spot and often can be beaten by other methods, like gradient boosting. There is no one algorithm to rule them all, just different algorithms for different problems and our job is to discover what works best on a given problem.
REPLY
Neel June 28, 2017 at 8:17 pm # Thanks for your response Jason! I appreciate your clarification.
REPLY
Jason Brownlee June 29, 2017 at 6:33 am # You’re welcome.
REPLY
Yukin July 27, 2017 at 6:29 pm # I am wondering that if I use a convolutional neural work in my train model, could I say it is deep learning?
REPLY
Jason Brownlee July 28, 2017 at 8:29 am # Yes.
REPLY
Suchithra August 4, 2017 at 1:43 pm # Sir , it’s a great review about deep learning My question is what is the difference between deep neural network and CNN. Is deep learning is applicable to quantitative data( tabular data) I made a deep neural network model for bulk quantitative data and get a better result than traditional neural method. What it means sir ? Is my deep learning technique right?
REPLY
Jason Brownlee August 4, 2017 at 3:45 pm # A CNN is a type of neural network. It can be made deep. Therefore, it is a type of deep neural network. Yes, neural nets require all input data to be tabular (vectorized). I cannot know if your model is right. Evaluate it carefully and compare it to other models.
REPLY
mera August 17, 2017 at 11:29 pm #
if we use hierarchal training algorithm such as we use unsupervised learning autoencoder with bottleneck (hidden layer, 10,2,10) for training then use the supervised learning with same autoencoder architecture ( hidden layer, 10,2,10) to tune the unsupervised model parameter (weights, bias). These training processes are performed separately. now the unsupervised autoencoder works as dimension reduction and extract features. does the supervised model work in the same way and extract the feature or just this step conducted in the unsupervised learning
REPLY
Jason Brownlee August 18, 2017 at 6:20 am # he supervised model will interpret the features and use them to make predictions.
REPLY
jayaram September 23, 2017 at 3:57 pm # can i say deeplerning == cnn, Do we have types in deep learning
REPLY
Jason Brownlee September 24, 2017 at 5:14 am # CNN is a type of deep learning. So is RNN and MLP.
REPLY
Sudha September 23, 2017 at 8:15 pm #
Sir, It is a good intro to deep learning. i’m planning to do phd in diagnosis of heart disease using deep learning. I have data’s of features. I don’t know how to classify those data. Can you please refer some material for numerical data classification using tensor flow.
REPLY
Jason Brownlee September 24, 2017 at 5:16 am # I would recommend following this process when working through a new predictive modeling problem: https://machinelearningmastery.com/start-here/#process
REPLY
mhamed kassem October 3, 2017 at 11:28 pm #
hi , i have started in my graduation project , and i am creating a medical system based on data comes from medical sensors , and i hope to use deep learning in detecting the disease or the health case based on that data , but i don’t know from where i should start in deep learning can you help me in doing it ?
REPLY
Jason Brownlee October 4, 2017 at 5:46 am # My best advice for getting started with deep learning in Python is here: https://machinelearningmastery.com/start-here/#deeplearning
REPLY
Cathy October 5, 2017 at 9:37 am # Hi Jason, thank you for the excellent overview. May I know how to apply deep learning in predicting adverse drug reactions, particularly in drug-drug interaction?
REPLY
Jason Brownlee October 5, 2017 at 5:21 pm # I recommend this process when starting on a new predictive modeling problem: https://machinelearningmastery.com/start-here/#process
REPLY
Sudha October 21, 2017 at 7:48 pm # Hi Jason, Can you please tell me the unsupervised deep learning algorithms available? Please refer some link to learn about it.
REPLY
Jason Brownlee October 22, 2017 at 5:18 am # Sorry, I don’t have material on unsupervised learning algorithms, I don’t find them useful in practice.
REPLY
naivebae October 31, 2017 at 2:56 pm #
What does “bigger models” mean? Are there more equations in the model? Are there more variables in the model? Are there more for loops? How exactly is the model “bigger”? What’s an example of a “not big model” and why is that worse? And what exactly is meant by the term “model” in this field? I’m still not clear on that. Like what exactly are the characteristics that make something a “model” and why is another general programming algorithm like, say, a loop that divides numbers successively to determine if an integer is prime, not a “model”? To me that sounds like a “model” for determining if a number is prime, so what is meant in this field by “model”? What are the inherent properties that make something a “model”? Is a model a type of algorithm? Is it a class in object-oriented design? What’s a “model” and what;s a “bigger model”?
REPLY
Jason Brownlee October 31, 2017 at 2:59 pm # Too many questions for one comment, sorry.
REPLY
naivebae October 31, 2017 at 4:30 pm #
Ok, let’s start by what exactly is meant by the term “model” in this field? Because I’m still absolutely not clear on what that means. If I write up some code to solve a problem, how do I know whether or not I’ve “built a model”?
REPLY
Jason Brownlee November 1, 2017 at 5:43 am # A model is often referred to as the weights + structure after the training algorithm has been run on data. It is the “program” that can make predictions on new data.
naivebae November 1, 2017 at 7:55 am # And then what’s the “bigger model” you refer to in this article? Are there more weights and more structure in the training algorithm? How is that achieved? Do you plug in more equations and more variables and decision parameters/whatever? How do you know what additional equations and parameters to plug in, and how do you know those are the right ones as opposed to others? Or does “bigger models” simply refer to running on more overall data, and the training algorithm is the same with the same amount of equations/variables/computations per training example?
Jason Brownlee November 1, 2017 at 4:13 pm # “Larger Models” refers to deeper (more layers) or wider (more neurons per layer), ultimately more representational capacity.
REPLY
Sukesh Kumar Ranjan November 3, 2017 at 9:10 am # Best summary for students to learn basics of Deep Learning. Thank you Jason
REPLY
Jason Brownlee November 3, 2017 at 2:16 pm # Thanks.
REPLY
Souha November 8, 2017 at 8:01 pm # Thank you very much. It is very good summary about deep learning. Could you give some algorithms used in deep learning , please.
REPLY
Jason Brownlee November 9, 2017 at 9:57 am # The three to focus on are: Multilayer Perceptron, Convolutional Neural Network and Long Short-Term Memory Network.
REPLY
chithambaram November 21, 2017 at 7:48 pm #
how deep learning can be applied in music ? is it possible to analyze existing music to compose new music by computers ?? If yes what type of algorithm should be used ?
REPLY
Jason Brownlee November 22, 2017 at 11:11 am # Yes, you could classify music or generate music.
REPLY
Javed Iqbal November 21, 2017 at 8:38 pm # Dear Dr Jason Brownlee, I really found this very useful and helpful for beginners to this domain.
REPLY
Jason Brownlee November 22, 2017 at 11:11 am # Thanks.
REPLY
Hamed December 6, 2017 at 4:46 pm #
I am familiar with machine learning and neural networks. My expertise is optimization and I am just interested in this field. What do you suggest as a good starting point? I prefer to learn it through experience and see how it works on different cases.
REPLY
Jason Brownlee December 7, 2017 at 7:50 am # I have some advice for getting started here that might help: https://machinelearningmastery.com/start-here/#deeplearning
REPLY
Hamed December 7, 2017 at 1:14 pm # Thanx a lot
REPLY
JD Maloy December 8, 2017 at 7:50 am # Do you think a deep-learning system could be “taught to read” in a manner similar to a young child? That is: 1. Visual input of the words on each page 2. Coordinated (i.e. synchronized) sound input of those words being read 3. Repeat/reinforce as needed by “reading” the same book multiple times 4. Expand the data set by “reading” a variety of books …with the ultimate goal & test being to present the system with a book it hasn’t “seen” before, and have it “read” that book via a synthesized voice. (Never mind the robotics to turn the pages, we can leave that part to a human assist.)
REPLY
Jason Brownlee December 8, 2017 at 2:27 pm # A general system of this form could be constructed, not sure about the “like a child” part though.
REPLY
nazek hassouneh December 11, 2017 at 4:08 am # i am nazek hassouneh i am a master student and my thesis in deep learning i have data set with class label i will use deep learning for classification data set is an excel format and will convert to CSV form data set with 49 attribute and 131,000 records how can i use deep learning i need tutorial for that i need specific nueral networks suitable for this data set
REPLY
Jason Brownlee December 11, 2017 at 5:34 am # Perhaps this tutorial will help you get started: https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/
REPLY
nazek hassouneh December 11, 2017 at 5:48 am # thank you very much
REPLY
Alexander January 23, 2018 at 11:05 pm # Very Usefull Article In Deep Learning
REPLY
Jason Brownlee January 24, 2018 at 9:56 am # Thanks Alexander.
Leave a Reply
Name (required)
Email (will not be published) (required)
Website
SUBMIT COMMENT
Welcome to Machine Learning Mastery Hi, I'm Jason Brownlee, Ph.D. My goal is to make practitioners like YOU awesome at applied machine learning. Read More
Finally Get Started With Deep Learning Sick of the fancy math and need for super computers? Looking for step-by-step tutorials? Want end-to-end projects? Get Started With Deep Learning in Python Today!
POPULAR
Time Series Prediction with LSTM Recurrent Neural Networks in Python with Keras JULY 21, 2016
Your First Machine Learning Project in Python Step-By-Step JUNE 10, 2016
Multivariate Time Series Forecasting with LSTMs in Keras AUGUST 14, 2017
How to Setup a Python Environment for Machine Learning and Deep Learning with Anaconda MARCH 13, 2017
Develop Your First Neural Network in Python With Keras Step-By-Step MAY 24, 2016
Sequence Classification with LSTM Recurrent Neural Networks in Python with Keras JULY 26, 2016
Time Series Forecasting with the Long Short-Term Memory Network in Python APRIL 7, 2017
Multi-Class Classification Tutorial with the Keras Deep Learning Library JUNE 2, 2016
Regression Tutorial with the Keras Deep Learning Library in Python JUNE 9, 2016
How to Grid Search Hyperparameters for Deep Learning Models in Python With Keras AUGUST 9, 2016
© 2018 Machine Learning Mastery. All Rights Reserved. Privacy | Contact | About