Learning Motorcar Learning: A Beginner's Journeying
I guide maintain been learning well-nigh motorcar learning in addition to deep learning (ML/DL) for the concluding year. I intend ML/DL is hither to stay. I don't intend this is a fad or bubble! Here is why:
Certainly at that spot is also a lot of hype well-nigh ML/DL. ML/DL proved feasible for specific sets of applications, in addition to it is a hyperbole to claim that the full general AI has arrived. We are far from it. But that is a adept thing, because nosotros volition guide maintain a lot of juicy problems to piece of work on.
So I am doubling downward on ML/DL.
Here are my outset impressions learning well-nigh ML/DL. ML/DL uses a really dissimilar toolkit in addition to approach than the distributed systems land I grew upwards in. I was initially surprised in addition to taken aback yesteryear the really experimental in addition to lawsuit in addition to fault nature of ML/DL. ML/DL is dealing amongst noisy/fuzzy/messy existent footing information in addition to naturally the land produced statistical in addition to probabilistic tools. Validation is alone via showing surgical physical care for on the examine set. The information ready is the king. Debugging is a mess, in addition to learning is really opaque. On the other hand, I actually similar the dynamism inwards the ML/DL area. There are a lot of resources in addition to platforms in addition to a lot of really interesting applications.
My involvement inwards ML/DL is inwards its interactions amongst distributed systems. I am non interested inwards writing image/text/speech processing applications. I learned well-nigh ML/DL to intend well-nigh 2 questions:
On a side note, if you lot similar to larn a chip well-nigh Ng's thinking physical care for in addition to his approach to life, creativity, in addition to failure, I recommend this interview. It is a really adept read.
I actually liked the outset iii weeks of Ng's course: Introduction in addition to Linear Regression, Linear Regression amongst multiple features, in addition to Logistic Regression in addition to regularization. But equally the course of written report went to logistic regression amongst nonlinear determination boundaries, I started to acquire overwhelmed amongst the amount of information in addition to complication. And equally the course of written report progressed to neural networks, I started to acquire lost. For example, I could non cast a adept mental model in addition to motion painting of forrard in addition to backward propagation inwards neural networks. So those parts didn't stick amongst me. (I was also non next the programming assignments well.)
I intend the job was that Ng was explaining the neural networks concepts inwards a general/generic way. That sounded likewise abstract to me. It mightiness guide maintain worked improve if he had settled on a pocket-size concrete occupation instance in addition to explained the concepts that way.
Recently, I started auditing a deep learning course of written report on Udacity amongst a Google Engineer, Vincent Vanhoucke. This course of written report offered a simpler introduction to deep learning. The course of written report started amongst multinomial logistic classification. Since I knew well-nigh logistic regression, I could follow this easily. I liked the softmax function, one-hot encoding, in addition to cross entropy ideas equally they are all really practical in addition to concrete concepts. The course of written report presented these amongst the occupation instance of MNIST missive of the alphabet classification for the outset 10 letters.
Then using the same MNIST example, the course of written report introduced rectified linear units (ReLu) equally a elementary agency of introducing nonlinearity in addition to showed how to chain multinomial logistic classification amongst ReLus to cook a deep network that solves the missive of the alphabet classification chore much better. This time, I was able to follow the forrard in addition to backward propagation ideas much better. Instead of explaining deep networks inwards a full general abstract way, this course of written report explained it inwards a ReLu-specific agency in addition to reusing the missive of the alphabet classification illustration built amongst logistical regression. (In Ng's degree ReLu came on calendar week 7, when introducing back upwards vector machines).
As a quick agency to overview Vincent's course of written report contents, you may sentinel this YouTube video from some other Google engineer. (Watch at 1.5x speed.) The utter uses the same MNIST illustration in addition to similar approach. But it doesn't acquire inwards explanation of forward/backward propagation in addition to deriving the ReLus, instead it focuses to a greater extent than on giving you lot introduction to TensorFlow skills equally good equally introducing the basic neural network concepts. Within 1 hour, the utter gets you lot learning well-nigh convolutional networks. The presentation is overnice in addition to slow to follow.
Reflecting back, it is interesting how much YouTube helped me to larn well-nigh ML/DL. I commonly similar reading papers improve than listening/watching videos, or perhaps that is because I am to a greater extent than accustomed to learning that way. But for learning well-nigh ML/DL, these YouTube videos guide maintain been really helpful. It looks similar lecturing is making a come upwards back.
Here is a bonus video of Andrew Ng talking well-nigh nuts in addition to bolts of applying deep learning. Ng is a peachy teacher, in addition to then it is slow to follow the concepts presented in addition to larn a lot from this talk.
In the coming days I promise to write brief summaries of the introductory ML/DL concepts I learned from these courses. In my outset brace posts on this, I computer program to follow the outset iii weeks of Ng's class. There are really adept course of written report notes of that course of written report here, in addition to I volition summarize fifty-fifty to a greater extent than briefly to advert the big ideas. Then I volition switch to Vanhoucke's course of written report to innovate the ideas inwards multinomial logistical regression in addition to the generalization to neural networks in addition to deep learning from there. I volition occupation the #mlbegin tag for the series. Let's encounter how that goes.
UPDATE (1/11/17): I wrote them up. Here are the introductory ML/DL concepts I learned from those courses:
- ML/DL has results. It is difficult to fence against success.
- ML/DL has been on the ascent organically since 1985 (with the backpropagation algorithms) in addition to went through some other stage of acceleration after 2005 (with the broad availability of big information in addition to distributed information processing platforms). The ascent of ML/DL is next a rising bend pattern, non the designing for a hyped ephemeral bubble. Since it grew gradually over many years, I am betting it volition hold out roughly for at to the lowest degree the same amount of time.
- ML/DL has been co-developed amongst applications. It has developed really much on the practise side amongst lawsuit in addition to error, in addition to its theory is withal lagging a chip behind in addition to is unable explicate many things. According to Nassim Taleb's heuristics ML/DL is antifragile.
- ML/DL has the marketplace position behind it. Big coin provides big incentive in addition to has been attracting a lot of smart people. This many smart people cannot hold out wrong.
Certainly at that spot is also a lot of hype well-nigh ML/DL. ML/DL proved feasible for specific sets of applications, in addition to it is a hyperbole to claim that the full general AI has arrived. We are far from it. But that is a adept thing, because nosotros volition guide maintain a lot of juicy problems to piece of work on.
So I am doubling downward on ML/DL.
Here are my outset impressions learning well-nigh ML/DL. ML/DL uses a really dissimilar toolkit in addition to approach than the distributed systems land I grew upwards in. I was initially surprised in addition to taken aback yesteryear the really experimental in addition to lawsuit in addition to fault nature of ML/DL. ML/DL is dealing amongst noisy/fuzzy/messy existent footing information in addition to naturally the land produced statistical in addition to probabilistic tools. Validation is alone via showing surgical physical care for on the examine set. The information ready is the king. Debugging is a mess, in addition to learning is really opaque. On the other hand, I actually similar the dynamism inwards the ML/DL area. There are a lot of resources in addition to platforms in addition to a lot of really interesting applications.
My involvement inwards ML/DL is inwards its interactions amongst distributed systems. I am non interested inwards writing image/text/speech processing applications. I learned well-nigh ML/DL to intend well-nigh 2 questions:
- How tin nosotros build improve distributed systems/architectures to improve the surgical physical care for of ML/DL systems/applications?
- How tin nosotros occupation ML/DL to build improve distributed systems?
How I went well-nigh learning ML/DL
In January, I started next Andrew Ng's motorcar learning course of written report at Coursera. (Alternatively, here is Ng's course of written report textile for CS 229 at Stanford.) After the kids went to sleep, I spent an hr each nighttime next Ng's degree videos. Andrew Ng has a overnice in addition to elementary agency of explaining ML concepts. He is a really adept teacher.On a side note, if you lot similar to larn a chip well-nigh Ng's thinking physical care for in addition to his approach to life, creativity, in addition to failure, I recommend this interview. It is a really adept read.
I actually liked the outset iii weeks of Ng's course: Introduction in addition to Linear Regression, Linear Regression amongst multiple features, in addition to Logistic Regression in addition to regularization. But equally the course of written report went to logistic regression amongst nonlinear determination boundaries, I started to acquire overwhelmed amongst the amount of information in addition to complication. And equally the course of written report progressed to neural networks, I started to acquire lost. For example, I could non cast a adept mental model in addition to motion painting of forrard in addition to backward propagation inwards neural networks. So those parts didn't stick amongst me. (I was also non next the programming assignments well.)
I intend the job was that Ng was explaining the neural networks concepts inwards a general/generic way. That sounded likewise abstract to me. It mightiness guide maintain worked improve if he had settled on a pocket-size concrete occupation instance in addition to explained the concepts that way.
Recently, I started auditing a deep learning course of written report on Udacity amongst a Google Engineer, Vincent Vanhoucke. This course of written report offered a simpler introduction to deep learning. The course of written report started amongst multinomial logistic classification. Since I knew well-nigh logistic regression, I could follow this easily. I liked the softmax function, one-hot encoding, in addition to cross entropy ideas equally they are all really practical in addition to concrete concepts. The course of written report presented these amongst the occupation instance of MNIST missive of the alphabet classification for the outset 10 letters.
Then using the same MNIST example, the course of written report introduced rectified linear units (ReLu) equally a elementary agency of introducing nonlinearity in addition to showed how to chain multinomial logistic classification amongst ReLus to cook a deep network that solves the missive of the alphabet classification chore much better. This time, I was able to follow the forrard in addition to backward propagation ideas much better. Instead of explaining deep networks inwards a full general abstract way, this course of written report explained it inwards a ReLu-specific agency in addition to reusing the missive of the alphabet classification illustration built amongst logistical regression. (In Ng's degree ReLu came on calendar week 7, when introducing back upwards vector machines).
As a quick agency to overview Vincent's course of written report contents, you may sentinel this YouTube video from some other Google engineer. (Watch at 1.5x speed.) The utter uses the same MNIST illustration in addition to similar approach. But it doesn't acquire inwards explanation of forward/backward propagation in addition to deriving the ReLus, instead it focuses to a greater extent than on giving you lot introduction to TensorFlow skills equally good equally introducing the basic neural network concepts. Within 1 hour, the utter gets you lot learning well-nigh convolutional networks. The presentation is overnice in addition to slow to follow.
Reflecting back, it is interesting how much YouTube helped me to larn well-nigh ML/DL. I commonly similar reading papers improve than listening/watching videos, or perhaps that is because I am to a greater extent than accustomed to learning that way. But for learning well-nigh ML/DL, these YouTube videos guide maintain been really helpful. It looks similar lecturing is making a come upwards back.
Here is a bonus video of Andrew Ng talking well-nigh nuts in addition to bolts of applying deep learning. Ng is a peachy teacher, in addition to then it is slow to follow the concepts presented in addition to larn a lot from this talk.
In the coming days I promise to write brief summaries of the introductory ML/DL concepts I learned from these courses. In my outset brace posts on this, I computer program to follow the outset iii weeks of Ng's class. There are really adept course of written report notes of that course of written report here, in addition to I volition summarize fifty-fifty to a greater extent than briefly to advert the big ideas. Then I volition switch to Vanhoucke's course of written report to innovate the ideas inwards multinomial logistical regression in addition to the generalization to neural networks in addition to deep learning from there. I volition occupation the #mlbegin tag for the series. Let's encounter how that goes.
UPDATE (1/11/17): I wrote them up. Here are the introductory ML/DL concepts I learned from those courses:
0 Response to "Learning Motorcar Learning: A Beginner's Journeying"
Post a Comment