Engineering Distributed Infrastructure

elective monographs

1000-213bPW, 1000-214bSIK, 1000-214bSOB

The course takes an engineer’s perspective on building the complex set of systems and services that together provide the public cloud. The cloud infrastructure is a complex distributed system with unique requirements: high availability, massive scale and having deep software stacks. We plan to show how these requirements influence the key design decisions (communication, scalability, resource management, data management) and reliability engineering (monitoring, testing). The course will be delivered by Google engineers whose day-to-day work involves designing, building and maintaining cloud infrastructure.

In contrast to teaching specific cloud technologies, our goal is to rather show principles driving the design of large-scale distributed systems. We plan to illustrate these principles using specific systems developed by your lecturers.

The course consists of lectures, exercises and a project.

Lectures and exercises will cover the following topics:

- Compute as a service: an introduction to the architecture of cloud computing

- Managing distributed resources, on scale: scheduling and autoscaling

- Communication within and outside the cloud: methods and APIs

- Designing for scalability

- Designing and managing systems for reliability and maintainability

- Monitoring

- Testing

- Managing distributed data

The project consists of designing, building and deploying a distributed application on a public cloud. We will suggest a few ideas for applications. The project is done in a team of 3 students. Each team will have an assigned Googler tutor who will help to scope the project; review design documents and grade the final solution.

Software Engineering at Google, Titus Winters, Tom Manshreck, Hyrum Wright, 2020, O'Reilly Media.

Site Reliability Engineering, Besty Beyer, Chris Jones, Jennifer Petoff, Niall Richard Murphy (Eds.), 2016, O'Reilly Media.

Additional bibliography (papers, technical documentation) will be given during the lectures.

Knowledge:

Students understand the issues of large-scale distributed computing systems.

Skills/Expertise:

Students can design and develop a complex, low-level component of a distributed computing infrastructure.

- Active participation in the exercises

- Project (in a 3-person team)

- Final exam

To pass, you need to actively attend exercises, complete the project and pass the final exam.