Statistical machine learning

elective monographs

The goal of the course is to introduce fundamental notions and statistical tools used in machine learning such as linear, logistic and multivariate regression, classifiers, dimension reduction methods, bayesian methods.

The detailed program

1. Explanatory Statistics (2-3 lessons)
1. Basic summary statistic (sample mean, median, sample variance, etc.)
2. Visualization of data (histogram, box-plots, kernel density estimation)
3. Principal component analysis
4. Clusterization, hierarchical clustering, k-means, k-medoids 
2. Statistical theory (4-5 lessons)
1. Basic definitions (statistical models, statistics, likelihood, etc.)
2. Estimation theory (maximum likelihood estimators, efficiency, mean square error, bias vs variance trade-off, confidence intervals)
3. Statistical hypothesis testing (type I and type II errors, power of a test, significance, p-value)
4. Problems with p-values (effect size, multiple hypotheses testing)
5. Bayesian inference (prior distribution, posterior distribution, Bayesian risk and Bayesian estimator, credible intervals)
6. Distance between probability measures (Kullback-Leibler divergence, total variation distance, etc.)
3. Simple regression and classification models (3-4 lessons)
1. Linear regression
2. Classification. Logistic regression, LDA, QDA
3. Cross-validation and bootstrap
4. Model selection and regularization. Lasso, ridge regression, forward-backwards procedure
4. Advanced models (3 lessons)
1. Tree-like models, bagging, random forests, boosting
2. Support vector machines
3. Non-linear models: splines, generalized additive models

1. Trevor Hastie, Robert Tibshirani, Jerome H., The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, Berlin

2. Andrew Ng, Machine Learning Yearning, https://github.com/ajaymache/machine-learning-yearning

Knowledge: the student

* has in-depth understanding of the branches of mathematics necessary to study machine learning (probability theory, statistics, multivariable calculus, and linear algebra) [K_W05];

* has based in theory and well organized knowledge of fundamental techniques of statistics used in modeling and data analysis [K_W07].

Abilities: the student is able to

* construct mathematical reasoning [K_U06];

* express problems in the language of mathematics [K_U07];

* apply techniques of modern statistical data analysis [K_U10].

Social competences: the student is ready to

* critically evaluate acquired knowledge and information [K_K01];

* recognize the significance of knowledge in solving cognitive and practical problems and the importance of consulting experts when difficulties arise in finding a self-devised solution [K_K02];

* think and act in an entrepreneurial way [K_K03].

Impact on the final grade: the final test 50%, two programming assignments 50%, in lab activity 10%.