CLASS INFORMATION:
Lectures: Tue 1:50 -- 3:40 pm
Location: Computer Science Bldg. room 1441

Instructor: Prof. M. Alex O. Vasilescu
Office Hours: Wed 1-2pm or by appointment

Prerequisites: Linear Algebra, Probability, or consent of the instructor.

Course Description:

This seminar emphasizes the fundamentals of statistical learning, pattern analysis and recognition. Topics covered include supervised and unsupervised learning, regression methods, classification methods, basic expansions and regularization, kernel methods, model assessment and selection, model inference and averaging, boosting, neural networks, support vector machines, multilinear (tensor) analysis, nearest-neighbor methods, and unsupervised clustering.

We will examine one or two papers per class. Students will be grouped into teams. Each team will be assigned papers to analyze, deliver a lecture and lead the classroom discussions. All the students are required to read every paper before it is discussed.  Each class will start by collecting questions from all the students about the topic of the day, followed by a student presentation, and then classroom discussions based on the collected questions.

There will be no exams or homeworks in this course. The grading will be based on a course project (50%), the quality of the seminar presentations delivered by each team(30%), class participation, and questions submitted at the start of each class (20%).

Lecture Schedule

Reference Textbooks:

• "Pattern Classification" by Richard O. Duda, Peter E. Hart, David G. Stork
• "The Elements of Statistical Learning: Data Mining, Inference, and Prediction" by Trevor Hastie, Robert Tibshirani and Jerome Friedman
• "Neural Networks for Pattern Recognition" by Christopher M. Bishop

1. Turk, M. & Pentland, A. (1991). "Eigenfaces for recognition" Journal of Cognitive Neuroscience, 3, 71-86.
   http://www.cs.ucsb.edu/~mturk/Papers/jcn.pdf



2. B. Moghaddam, W. Wahid, and A. Pentland "Beyond eigenfaces: Probabilistic matching for face recognition", In Proc. of International Conf. on Automatic Face and Gesture Recognition, pages 30--35, Nara, Japan, April 1998.
   http://citeseer.ist.psu.edu/moghaddam98beyond.html

___________________________________________________________________________________


3. Hyvarinen, Aapo "Independent Component Analysis: Algorithms and Applications" in Neural Networks, 13(4-5):411-430, 2000.
   http://www.cis.hut.fi/aapo/papers/IJCNN99_tutorialweb/



4. Bartlett M.S., Movellan J.R., Sejnowski T.J, `Face Recognition by Independent Component Analysis', IEEE Transactions on Neural Networks 13 (6) (2002) 1450-- 1464.
   http://citeseer.ist.psu.edu/bartlett02face.html

___________________________________________________________________________________


5. M. Brand. Incremental singular value decomposition of uncertain data with missing values. In A. Heyden, G. Sparr, M. Nielsen, and P. Johansen, editors, Seventh Europian Conference on Computer Vision (ECCV), volume 1, pages 707--720. Springer Verlag (LNCS 2350), 2002.
   http://citeseer.ist.psu.edu/665917.html
   
   
6. M. Brand. Fast online SVD revisions for lightweight recommender systems. In Proc. SIAM International Conference on Data Mining, 2003.
   Paper - pdf.

___________________________________________________________________________________


7. Lee, D.D.; Seung, H.S. Learning the parts of objects by non-negative matrix factorization. Nature, vol.401, no.6755, Macmillan Magazines, 21 Oct. 1999. p.788-91.

___________________________________________________________________________________


8. Belhumeur N., Hespanha J. and Kriegman D., "Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection", Proc. ECCV, 45--58, 1996.
   http://citeseer.ist.psu.edu/belhumeur96eigenfaces.html

___________________________________________________________________________________


9. Vasilescu M.A.O., Terzopoulos D., "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark, May, 2002, in Computer Vision -- ECCV 2002, Lecture Notes in Computer Science, Vol. 2350, A. Heyden et al. (Eds.), Springer-Verlag, Berlin, 2002, 447-460.
   Abstract | Full Article in PDF (882KB)
   
   
10. Vasilescu M.A.O., Terzopoulos D., `Multilinear Independent Components Analysis', Proc. Computer Vision and Pattern Recognition Conf. (CVPR '05), 547-553 vol.1, 2005.
    Paper (1,027 KB - .pdf)



11. Vasilescu M.A.O., Terzopoulos D., `Multilinear Projection for Recognition: Extending the Tensor Framework', Proc. International Conference on Computer Vision (ICCV '07), 2007 (to appear).
    ___________________________________________________________________________________
    
    
12. Cootes, T.F.; Edwards, G.J.; Taylor, C.J. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, June 2001, vol.23, (no.6):681-5.

___________________________________________________________________________________


13. R. Bro, PARAFAC: Tutorial & applications, in: 2nd Internet Conf. in Chemometrics (INCINC'96), Chemometrics Intell. Lab. Syst., vol. 38, 1997, pp. 149–171 (special issue)
    http://www.models.kvl.dk/users/rasmus/presentations/parafac_tutorial/paraf.htm
    
    
14. R.A. Harshman, Foundations of the PARAFAC procedure: model and conditions for an “explanatory” multi-mode factor analysis, UCLA Working Papers in Phonetics 16(1) (1970)

___________________________________________________________________________________


Kruskal, J.B., "Rank, decomposition, and Uniqueness for 3-Way and N-Way Decomposition", Multiway Data Analysis, R. Coppi and S.Bolasco (Eds.), Elsevier Science Publishers - North-Holland 1989.


16. Blinn, J.F., "How to solve a cubic equation. Part 1. The shape of the discriminant", Computer Graphics and Applications, IEEE, May-June 2006, vol. 26, nr. 3, pgs: 84- 93
    ___________________________________________________________________________________
    
    
17. Schoelkopf, B.; Smola, A.; Mueller, K.-R. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 1 July 1998, vol.10, (no.5):1299-319.



18. K.-R. Muller, S. Mika, G. Rätsch, K. Tsuda, and B. Scholkopf. "An introduction to kernel-based learning algorithms," IEEE Neural Networks, 12(2):181-201, May 2001.



19. B. Schoelkopf, S. Mika, C.J.C. Burges, P. Knirsch, K.-R. Mueller, G. Raetsch, and A.J. Smola. Input space vs. feature space in kernel-based methods. IEEE Transactions on Neural Networks, 10(5):1000-1017, September 1999.



20. S. Mika, B. Schoelkopf, A.J. Smola, K.-R. Mueller, M. Scholz, and G. Raetsch. Kernel PCA and de-noising in feature spaces. In M.S. Kearns, S.A. Solla, and D.A. Cohn, editors, Advances in Neural Information Processing Systems 11, pages 536-542. MIT Press, 1999.
    
    
21. C. J. C. Burges. A Tutorial on Support Vector Machines for Pattern Recognition. Knowledge Discovery and Data Mining, 2(2), 1998



22. Tutorials on Kernel Methods and SVM

___________________________________________________________________________________


23. Roweis, S.T.; Saul, L.K. Nonlinear dimensionality reduction by locally linear embedding. Science, 22 Dec. 2000, vol.290, (no.5500):2323-6.



24. Tenenbaum, J.B.; de Silva, V.; Langford, J.C. A global geometric framework for nonlinear dimensionality reduction. Science, 22 Dec. 2000, vol.290, (no.5500):2319-23.
    ___________________________________________________________________________________
    
    
25. D. Comaniciu, P. Meer: Mean shift analysis and applications. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece, 20-27 Sept. 1999.



26. Subbarao Raghav and Meer Peter, Nonlinear Mean Shift for Clustering over Analytic Manifolds,  CVPR 2006.

___________________________________________________________________________________


27. Using Multiple Segmentations to Discover Objects and their Extent in Image Collections, Bryan Russell, Alexei Efros, Josef Sivic, William Freeman, and Andrew Zisserman, CVPR'06.