Machine learning

elective monographs

Machine learning usually refers to a system that is capable to learn from experience, analytical observation and historical data. This capacity should result in a system that can continuously self-improve and thereby offer increased efficiency and effectiveness. This course will be an introduction to the basics of machine learning. We will study multiple machine learning models including decision trees, neural networks, Bayesian learning, instance-based learning, and genetic algorithms. In doing so, we will begin to understand some of the issues and challenges facing attempts at machine learning-generalization, bias, overfitting, model selection, feature selection and learnability-while being exposed to the pragmatics of implementing machine learning systems.

1. Introduction: Preliminaries and basic notions in machine learning

2. Supervised learning methods: Classification problem, classifier evaluation methods, basic classification algorithms: Na?ve Bayes, KNN, decision rules, decision trees, decision forest. Artificial Neural Networks.

3. Learning function and concept approximation: "Gradient descent" and "Back Probagation" algorithms, logistic regression, SVM classifiers.

4. Computational Learning Theory (COLT): PAC model in learning theory, VC dimension, bagging & boosting methods. Multiclass to binary reduction, cost-sensitive learning, ranking learning;

5. Unsupervised Learning: Hierarchical Clustering, K-means, Expectation Maximization (EM) method. Principal component analysis: PCA. MDS. pPCA. Independent component analysis: ICA.

6. Reinforcement learning: MDP (Markov decision processes), Bellman equations, TD(?) learning (Temporal-difference learning) and Q-learning

1. Bishop, "Pattern Recognition and Machine Learning", 2007

2. Hastie, Tibshirani and Friedman, "Elements of Statistical Learning: Data Mining, Inference and Prediction", 2001

3. MacKay, "Information Theory, Inference, and Learning Algorithms", 2003.

4. Mitchell, "Machine Learning", 1997