Interactive Machine Learning

elective monographs

Statistical machine learning 1000-317bSML

Data mining 1000-2M03DM
Machine learning 1000-2N09SUS

During the course, techniques of interactive data mining and constructing machine learning models will be presented. In particular, active learning and visual data mining techniques will be discussed.

The main topics discussed during the course:

1. Active learning.

2. Selection of examples for effective model training.

3. Model updates and incremental learning.

4. Active learning using batches of examples:

a. Ensuring batch variety.

b. Measures of the representativeness of examples.

5. Optimizing the selection of examples for experts.

6. Methods of achieving a consensus between voters on data labels.

7. Uncertainty of models vs. example informativeness.

8. Drift of concepts and interactive adaptation of models.

9. Interactive discovery of anomalies in data.

10. Counterfactual explanations in the context of uncertainty reduction.

11. Semi-supervised learning - using models to generate training data on unlabeled parts of data.

12. Interactive feature engineering.

13. Visual data mining.

14. Selected issues in the field of life-long learning.

1. B. Settles: Active Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, Morgan & Claypool Publishers 2012

2. Y. Xu, F. Sun, X. Zhang: Literature survey of active learning in multimedia annotation and retrieval. In Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service (ICIMCS '13). ACM, New York, NY, USA, 237–242. 2013.

3. S.-J. Huang, R. Jin, and Z.-H. Zhou. Active learning by querying informative and representative examples. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(10):1936–1949, Oct. 2014.

4. S. Sinha, S. Ebrahimi, and T. Darrell. Variational adversarial active learning. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, Oct. 2019.

5. R. Khan, M. Gubanov: WebLens: Towards Interactive Large-scale Structured Data Profiling CIKM 2020

6. H. H. Aghdam, A. Gonzalez-Garcia, A. Lopez, and J. Weijer. Active learning for deep detection neural networks. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, Oct. 2019.

After completing the course, participants will gain a broad understanding of active

learning techniques and skills in the interactive construction of ML models. In

particular:

Knowledge and skills:

1. Student knows the most important active learning techniques.

2. Student knows the basic methods for selecting a subset of data instances for labeling.

3. Student knows the basic techniques for updating ML models based on new data.

4. Student can evaluate the effectiveness of active learning techniques.

5. Student knows the basic methods of establishing consensus in a case of contradicting instance labels provided by many experts.

6. Student knows the basic techniques for identifying anomalies in data.

7. Student is familiar with the techniques of interactive data mining and visual data analysis.

8. Student knows the most important current trends in the field of active learning.

Competences:

1. Student is able prepare a report on the effectiveness of an active learning system.

2. Student can present the results of performed interactive data analysis.

3. Student can design a system for acquiring knowledge from domain experts to train and update predictive models.

The final grade is determined on the basis of the sum of the points from the

laboratory and the exam.

Additionally, students of the doctoral school can pass a subject by preparing a special project related to the practical use of active learning.