CSE 692






Stony Brook University (SUNY):

   Statistical Learning - CSE 692

Media Coverage

Adv. Topics in Statistical Learning: CSE 692

Fall 2007

Lectures: Tue 1:50 -- 3:40 pm
Location: Computer Science Bldg. room 1441

Instructor: Prof. M. Alex O. Vasilescu
Office Hours: Wed 1-2pm or by appointment

Prerequisites: Linear Algebra, Probability, or consent of the instructor.

Course Description:

    This seminar emphasizes the fundamentals of statistical learning, pattern analysis and recognition. Topics covered include supervised and unsupervised learning, regression methods, classification methods, basic expansions and regularization, kernel methods, model assessment and selection, model inference and averaging, boosting, neural networks, support vector machines, multilinear (tensor) analysis, nearest-neighbor methods, and unsupervised clustering.

    We will examine one or two papers per class. Students will be grouped into teams. Each team will be assigned papers to analyze, deliver a lecture and lead the classroom discussions. All the students are required to read every paper before it is discussed.  Each class will start by collecting questions from all the students about the topic of the day, followed by a student presentation, and then classroom discussions based on the collected questions.

    There will be no exams or homeworks in this course. The grading will be based on a course project (50%), the quality of the seminar presentations delivered by each team(30%), class participation, and questions submitted at the start of each class (20%).

Lecture Schedule

Reference Textbooks:

  • "Pattern Classification" by Richard O. Duda, Peter E. Hart, David G. Stork
  • "The Elements of Statistical Learning: Data Mining, Inference, and Prediction" by Trevor Hastie, Robert Tibshirani and Jerome Friedman
  • "Neural Networks for Pattern Recognition" by Christopher M. Bishop
Partial Reading List:
  1. Turk, M. & Pentland, A. (1991). "Eigenfaces for recognition" Journal of Cognitive Neuroscience, 3, 71-86.

  2. B. Moghaddam, W. Wahid, and A. Pentland "Beyond eigenfaces: Probabilistic matching for face recognition", In Proc. of International Conf. on Automatic Face and Gesture Recognition, pages 30--35, Nara, Japan, April 1998.

  3. ___________________________________________________________________________________

  4. Hyvarinen, Aapo "Independent Component Analysis: Algorithms and Applications" in Neural Networks, 13(4-5):411-430, 2000.

  5. Bartlett M.S., Movellan J.R., Sejnowski T.J, `Face Recognition by Independent Component Analysis', IEEE Transactions on Neural Networks 13 (6) (2002) 1450-- 1464.

  6. ___________________________________________________________________________________

  7. M. Brand. Incremental singular value decomposition of uncertain data with missing values. In A. Heyden, G. Sparr, M. Nielsen, and P. Johansen, editors, Seventh Europian Conference on Computer Vision (ECCV), volume 1, pages 707--720. Springer Verlag (LNCS 2350), 2002.

  8. M. Brand. Fast online SVD revisions for lightweight recommender systems. In Proc. SIAM International Conference on Data Mining, 2003.
    Paper - pdf.

  9. ___________________________________________________________________________________

  10. Lee, D.D.; Seung, H.S. Learning the parts of objects by non-negative matrix factorization. Nature, vol.401, no.6755, Macmillan Magazines, 21 Oct. 1999. p.788-91.

  11. ___________________________________________________________________________________

  12. Belhumeur N., Hespanha J. and Kriegman D., "Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection", Proc. ECCV, 45--58, 1996.

  13. ___________________________________________________________________________________

  14. Vasilescu M.A.O., Terzopoulos D., "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark, May, 2002, in Computer Vision -- ECCV 2002, Lecture Notes in Computer Science, Vol. 2350, A. Heyden et al. (Eds.), Springer-Verlag, Berlin, 2002, 447-460.
    Abstract | Full Article in PDF (882KB)

  15. Vasilescu M.A.O., Terzopoulos D., `Multilinear Independent Components Analysis', Proc. Computer Vision and Pattern Recognition Conf. (CVPR '05), 547-553 vol.1, 2005.
    Paper (1,027 KB - .pdf)

  16. Vasilescu M.A.O., Terzopoulos D., `Multilinear Projection for Recognition: Extending the Tensor Framework', Proc. International Conference on Computer Vision (ICCV '07), 2007 (to appear).

  17. Cootes, T.F.; Edwards, G.J.; Taylor, C.J. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, June 2001, vol.23, (no.6):681-5.

  18. ___________________________________________________________________________________

  19. R. Bro, PARAFAC: Tutorial & applications, in: 2nd Internet Conf. in Chemometrics (INCINC'96), Chemometrics Intell. Lab. Syst., vol. 38, 1997, pp. 149–171 (special issue)

  20. R.A. Harshman, Foundations of the PARAFAC procedure: model and conditions for an “explanatory” multi-mode factor analysis, UCLA Working Papers in Phonetics 16(1) (1970)

  21. ___________________________________________________________________________________

  22. Kruskal, J.B., "Rank, decomposition, and Uniqueness for 3-Way and N-Way Decomposition", Multiway Data Analysis, R. Coppi and S.Bolasco (Eds.), Elsevier Science Publishers - North-Holland 1989.

  23. Blinn, J.F., "How to solve a cubic equation. Part 1. The shape of the discriminant", Computer Graphics and Applications, IEEE, May-June 2006, vol. 26, nr. 3, pgs: 84- 93

  24. Schoelkopf, B.; Smola, A.; Mueller, K.-R. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 1 July 1998, vol.10, (no.5):1299-319.

  25. K.-R. Muller, S. Mika, G. Rätsch, K. Tsuda, and B. Scholkopf. "An introduction to kernel-based learning algorithms," IEEE Neural Networks, 12(2):181-201, May 2001.

  26. B. Schoelkopf, S. Mika, C.J.C. Burges, P. Knirsch, K.-R. Mueller, G. Raetsch, and A.J. Smola. Input space vs. feature space in kernel-based methods. IEEE Transactions on Neural Networks, 10(5):1000-1017, September 1999.

  27. S. Mika, B. Schoelkopf, A.J. Smola, K.-R. Mueller, M. Scholz, and G. Raetsch. Kernel PCA and de-noising in feature spaces. In M.S. Kearns, S.A. Solla, and D.A. Cohn, editors, Advances in Neural Information Processing Systems 11, pages 536-542. MIT Press, 1999.

  28. C. J. C. Burges. A Tutorial on Support Vector Machines for Pattern Recognition. Knowledge Discovery and Data Mining, 2(2), 1998

  29. Tutorials on Kernel Methods and SVM

  30. ___________________________________________________________________________________

  31. Roweis, S.T.; Saul, L.K. Nonlinear dimensionality reduction by locally linear embedding. Science, 22 Dec. 2000, vol.290, (no.5500):2323-6.

  32. Tenenbaum, J.B.; de Silva, V.; Langford, J.C. A global geometric framework for nonlinear dimensionality reduction. Science, 22 Dec. 2000, vol.290, (no.5500):2319-23.

  33. D. Comaniciu, P. Meer: Mean shift analysis and applications. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece, 20-27 Sept. 1999.

  34. Subbarao Raghav and Meer Peter, Nonlinear Mean Shift for Clustering over Analytic Manifolds,  CVPR 2006.

  35. ___________________________________________________________________________________

  36. Using Multiple Segmentations to Discover Objects and their Extent in Image Collections, Bryan Russell, Alexei Efros, Josef Sivic, William Freeman, and Andrew Zisserman, CVPR'06.