Alan Lee's List: ml-class

[Basics]

 

• Partial derivatives intuition. You can't ask about the slope of the surface because there's an infinite number of tangent lines at any point on the surface, so you have to "pick a direction" which is exactly what partial derivatives do: http://www.khanacademy.org/video/partial-derivatives

 

• The gradient descent intuition. The gradient (in 3d space) tells you the direction in (x,y) plane you should travel in order to get the maximum slope in the z dimension: http://www.khanacademy.org/video/gradient-1

 

• Linear Regression intuition Find out, why the function is mean squared rather that just sum of residuals or something else http://www.youtube.com/watch?v=VeXdn7YMqvY

 

• Linear Regression derivation Find out, how to derive the formula presented by professor Andrew Ng http://www.youtube.com/watch?v=hWUD57T3-Qg

 

• Gradient descent discussions on Math.SE, a Q&A site for people studying math at any level & professionals in related fields: http://math.stackexchange.com/search?q=%22gradient+descent%22

 

[Advanced]

 
• Practical usage of Linear Regression. Sampling data from Performance event counters: http://comparch.doc.ic.ac.uk/publications/files/qw09smart.pdf
 
• GC method which is reasonably more complicated but gives better performance in the vicinity of the minimum if that has the shape of a long, narrow valley: http://www.academicearth.org/lectures/recap-conjugate-gradient-method with slides at http://www.stanford.edu/class/ee364b/lectures/conj_grad_slides.pdf