Fundamental concepts

• Statistical Learning/Pattern Recognition
• Features
• Classification
• Regression
• Nonparametric regression/density estimation
• Parameter Estimation
• Model selection
• Independence Diagram
• Active learning
• Reinforcement learning
• No free lunch

Feature extraction techniques

• Discriminant Analysis
• Principal Component Analysis
• Principal curve
• Factor analysis
• Independent Component Analysis
• Clustering
• K-means
• Jarvis&Patrick clustering
• Feature selection

Statistical models

• Linear regression
• Basis function regression
• Gaussian process regression
• Radial Basis Function regression
• Generalized linear model
• Logistic regression
• Gaussian process classifier
• Feed-forward neural network regression
• Feed-forward neural network density model
• Additive regression
• Projection pursuit regression
• Robust regression
• Independent Feature Model (`Naive Bayes')
• Linear classifier
• Generalized linear classifier
• Support Vector Machine
• Finite mixture model
• Mixture of subspaces
• Coupled mixture model
• Markov chain
• Autoregression
• Hidden Markov Model
• Autoregressive Hidden Markov Model
• Input/Output Hidden Markov Model
• Factorial Hidden Markov Model
• Hidden Markov Decision Tree
• Switching Hidden Markov Model
• Hidden Markov Model with Duration
• Hidden Markov Segment Model
• Linear Dynamical System
• General dynamical system
• Mixture of Experts
• Decision tree
• Markov random field
• Simultaneous autoregression
• Hidden Markov random field
• Constrained mixture model
• Constrained Hidden Markov Model
• Coupled Hidden Markov Model
• Stochastic context-free grammar
• Stochastic program

Parameter estimation techniques

• Maximum likelihood
• Maximum A Posteriori
• Unbiased estimation
• Predictive estimation
• Minimum Message Length
• Bootstrapping
• Bagging
• Monte Carlo integration
• Regularization
• Expectation-Maximization (EM)
• Variational bound optimization
• Variational bound integration
• Jensen bound integration
• Expectation Propagation
• Newton-Raphson
• Iteratively Reweighted Least Squares
• Back-propagation
• Backfitting
• Kalman filtering
• Extended Kalman filtering
• Relaxation labeling
• Deterministic annealing
• Boosting
• Empirical Risk Minimization

Model selection techniques

• Cross-validation
• Bayesian model selection
• Minimum Message Length model selection
• Structural Risk Minimization
• Other model selection criteria

Nonparametric modeling

• Nearest-neighbor density estimation
• Nearest-neighbor classification
• Nearest-neighbor regression
• Kernel density estimation
• Locally weighted regression