Probabilistic and graph models of causality

elective monographs

We welcome undergraduate and graduate students interested in issues at the intersection of mathematics and the philosophy of science. Elementary calculus of probability is sufficient to understand the lecture. It is desirable - but not necessary - to have heard a lecture on Bayesian networks and be familiar with elements of information theory.

The lecture will be devoted to dynamic models of causality. They consist of graphs whose vertices are stochastic processes. We will summarize the results on the criteria for determining independence in such models.

We will discuss the concept of intervention (controlled experiment) and the problem of predicting the effect of the intervention on the basis of observational data. We will present applications of information theory to quantification of the strength of dependence in causality models.

One of the promising directions for the development of machine learning is looking for causal relationships between variables, instead of limiting ourselves to predicting variables.

The lecture will consider dynamic models of causality in the form of directed graphs whose vertices are stochastic processes (with discrete or continuous time). The edges of the graph describe the causal relationships between the components of the model. Compared to the classic models pioneered by J.Pearl in the 1980s, the difference lies in the explicit consideration of the time factor. This allows for feedback modeling by allowing cycles in the graph.

1) In the first part of the lecture, we will summarize the results characterizing the conditional independence of subsets of random variables in terms of graph properties (for example, how can you describe the conditional independence of the future of variable Y from the past of variable X having information about the past of variables Y and Z?). These types of results are quite well studied and described, although we will also present some recent additions.

2) The second leading topic of the lecture will be the concept of intervention, which is at the heart of thinking about causality. Intervention (a controlled experiment) means that the experimenter sets the values of certain variables and observes the effect of this manipulation on other variables. This is essentially what the definition of causality is all about, but very often such a controlled experiment is just a thought experiment (it is impossible or difficult to actually carry out for technical, ethical or economic reasons). The problem arises: how and when can we predict the results of an imaginary intervention on the basis of observational data only.

3) The third topic of the lecture will concern the connection between causality models and information theory. The idea is to move from testing (conditional) independence to quantifying (conditional) dependence using tools such as entropy or mutual information. In the literature, this type of problem goes under the name of "directed information theory." Results on quantifying the effects of interventions in the language of information theory are scarce, sometimes with controversial interpretations or simply containing errors. The lecture will provide a critical review of existing theory.

Although the literature on graph causality models is extensive, the range of theory chosen as the topic of our

lecture does not (as far as we know) have a separate monograph. The lecture will therefore be based on a few selected original articles.

ad 1): Graphical Models for Composable Finite Markov Processes.

Vanessa Didelez, Scandinavian Journal of Statistics, Vol. 34, No. 1 (March 2007), pp. 169-185

ad 1): Graphical Models for Marked Point Processes Based on Local Independence. Vanessa Didelez, Journal of the Royal Statistical Society. Series B (Statistical Methodology), Vol. 70, No. 1 (2008), pp. 245-264

ad 1) Local Dependence Graphs for Discrete Time Processes. Wojciech Niemiro and Łukasz Rajkowski, Proceedings of Machine Learning Research vol 213:1–19, 2023 (2nd Conference on Causal Learning and Reasoning)

ad 1), 2): Causal Reasoning in Graphical Time Series Models.

Michael Eichler and Vanessa Didelez, in: Proceedings of UAI 2007 (Uncertainty in Artificial Intelligence) pp. 109-116.

ad 3): The relation between Granger causality and directed information theory: a review, Pierre-Olivier Amblard and Olivier J.J. Michel (Entropy 15(1), 113-143, 2013)

ad 3): Information Theoretic Causal Effect Quantification, Aleksander Wieczorek and Volker Roth (Entropy, 21, 975; 2019)

Completion of exercises/tutorials will be based on the preparation of a micro-paper presented in class and containing a fragment of material supplementing the lecture (a selected part of some article).

The lecture ends with an oral exam in the form of a free conversation.