Idea Transcript
EECS 545: Machine Learning University of Michigan, Fall 2013 Instructor: Clayton Scott (clayscot) Classroom: FXB 1109 Time: TTh 10:30--12:00 Office: 4433 EECS Office hours: Mon. 1-4 PM or by appointment GSI: Robert Vandermeulen (rvdm) GSI office hours: Tuesday 2-4 PM and Thursday 2-3 PM, EECS 2420 Final Projects from Fall 2011 Final Projects from Fall 2009 Final Projects from Fall 2007 Required text: None. I will share my lecture notes prior to each lecture. Primary recommended text: Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, Second Edition (available online). Other recommended texts: Duda, Hart, and Stork, Pattern Classification, Wiley, 2001. Bishop, Pattern Recognition and Machine Learning, Springer, 2006. Sutton and Barto, Reinforcement Learning: An Introduction, MIT Press, 1998 Additional references Devroye, Gyorfi, and Lugosi, A Probabilistic Theory of Pattern Recognition, Springer, 1996. Scholkopf and Smola, Learning with Kernels, MIT Press, 2002 Mardia, Kent, and Bibby, Multivariate Analysis, Academic Press, 1979 (good for PCA, MDS, and factor analysis). Boyd and Vandenberghe, Convex Optimization, Cambridge University Press, 2004 Machine learning bibliography Prerequisites: (the current formal prerequisite is currently listed as EECS 492, Artificial Intelligence, but this is inaccurate) Probability: jointly distributed random variables, multivariate densities and mass functions, expectation, independence, conditional distributions, Bayes rule, the multivariate normal distribution Linear algebra: rank, nullity, linear independence, inner products, orthogonality, positive (semi-) definite matrices, eigenvalue decompositions. It is expected that students will have a good working knowledge of these topics. Students with most but not all of this background should be able to catch up during the semester with some additional effort. Topics: Statistical machine learning Supervised Learning Nearest neighbor classification The Bayes classifier Linear discriminant analysis Logistic regression Naive Bayes Separating hyperplanes Least squares linear regression Locally linear regression Nonlinear feature maps Regularization Inner product kernels and the kernel trick Kernel ridge regression Constrained optimization Support vector machines Unsupervised Learning Principal component analysis K-means clustering The EM algorithm for Gaussian mixture models Kernel density estimation Additional Topics (depending on student and instructor preferences) Model selection and error estimation Feature selection Spectral clustering Multidimensional scaling Nonlinear dimensionality reduction Decision trees Ensemble methods Boosting Hierarchical clustering Neural networks Gaussian processes Reproducing kernel Hilbert spaces Multitask and Transfer Learning, Group Sparsity, Matrix Completion, Robust PCA, Multiclass SVM Learning theory Grading: Homework: 35% Midterm exam: 30%, Thursday November 7, 6-9 PM, location TBA. Final project: 35% Homeworks: About four or five homeworks will be assigned before the midterm. Applications will be developed through Matlab programming exercises, including face recognition, spam filtering, handwritten digit recognition, image compression, and image segmentation. Most or all assignments will involve some computer programming. MATLAB will serve as the official programming language of the course. I will sometimes provide you with data, fragments of code, or suggested commands, in MATLAB. Exam: Thursday, November 7, 6-9 PM. Collaboration of any form will not be allowed. Allowed materials will be specified in advance of the exam. Please notify me the first week of class if you have a conflict. Final Project: There will be a final project. Groups will be allowed. The project must explore a methodology or application not covered in the lectures. Project guidelines and parameters will be announced at a later date, and may depend on the final enrollment of the course. Collaboration on homeworks: Each student will prepare the final write-up of his or her homework solutions without reference to any other person or source, aside from the student's own notes or scrap work. Students may consult classmates for the purpose of brainstorming, but not for obtaining the details of solutions. Under no circumstances may you copy solutions or code from a classmate or other source. Computer use in class: You may use your computer in class for note taking or note viewing, but otherwise please refrain from using computers or personal electronic devices during class, as these are distracting to me and your classmates. Honor Code All undergraduate and graduate students are expected to abide by the College of Engineering Honor Code as stated in the Student Handbook and the Honor Code Pamphlet. Students with Disabilities Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential.