2024-2025 Sem I

This course introduces cloud infrastructure. Students should feel more comfortable with building cloud services after having done this course.

Course Information

  • Prerequisites: COL331 or equivalent.

    Note: The course includes programming assignments and thus expects proficiency with systems programming and debugging.

  • Credits: 3-0-2
  • Slot: AC, Tuesdays and Fridays 2-3:15pm in LH521.
  • TAs:
    • TBD
  • Reading material: There is no textbook for the course. Most lectures will link to more reading material. Lecture notes can be found here and here.
  • Acknowledgements: Thanks to Robert T. Morris, MIT and Mythilli Vutukuru, IITB; parts of this course have been inspired by courses made available by them.

Grading criteria

  • 30% programming assignments
  • 20% project
  • 10% quizzes
  • 20% minor exam
  • 20% major exam

Supporting systems

  • Programming assignments are to be done on Baadal. You will need VPN access to IITD network!
  • Discussions should be done on Piazza.

Policies

  • Audit criteria: 40% or more marks in total. 40% or more marks in major+minor exams.
  • Ethics: Please re-read IITD honour code. Cheating will get an F in the course. Why should I not cheat?
  • Late policy: To help you cope with unexpected emergencies, you can hand in your Labs solutions late, but the total amount of lateness summed over all the lab deadlines must not exceed 72 hours. You can divide up your 72 hours among the labs however you like; you don’t have to ask or tell us. You can only use late hours only for Labs.
  • There will be no make up quizzes. We will count the scores from your best (n-1) quizzes where n is the total number of quizzes.

Tentative topics

Computation:

  • Translate existing programs to distributed system. (Distributed shared memory)
  • Batch computation (MapReduce, Spark), streaming computation (Spark streaming, Flink, Google Dataflow), ML training (Tensorflow)
  • The problem of late data in streaming computation (Millwheel, Google dataflow): watermarks, triggers, windows.
  • Fault tolerance strategies: re-run deterministic idempotent functions (MapReduce, Spark), asynchronous consistent checkpoints (Flink), inconsistent checkpoints (TensorFlow).
  • Straggler mitigation, scalability, locality, etc.

Storage:

  • PACELC theorem: If partitioned, choose between availability and consistency, else choose between latency and consistency.
  • CP systems:
    • Linearizability. Raft: quorums, leader election.
    • Serializability. Google Spanner: distributed transactions, TrueTime, hybrid logical clocks.
  • AP systems:
    • Amazon dynamo: eventual consistency, hashing, gossip protocols, dotted version vectors, conflict-free replicated data types (CRDTs)
  • Somewhere between CP and AP
    • Google file system
    • Zookeeper
    • RedBlue consistency

Hardware-assisted virtualization:

  • CPU virtualization: KVM, Popek-Goldberg theorem
  • Memory virtualization: 2-D page tables

Disclaimer: Actual course contents may differ depending on student interest. Reach out to the instructor as soon as possible if there is a particular interest in a topic.

Tentative Schedule

Week Tuesday Friday Sunday
1 23 Jul
LEC 1: Introduction.
26 Jul
LEC 2:.
 
2 30 Jul
LEC 3:
2 Aug
LEC 4:
 
3 6 Aug
LEC 5:
9 Aug
LEC 6:
 
4 13 Aug
tuesday is thursday. no class
16 Aug
no class day
 
5 20 Aug
LEC 7:
23 Aug
LEC 8:
 
6 27 Aug
LEC 9:
30 Aug
LEC 10:
 
7 3 Sep
LEC 11:
6 Sep
LEC 12:
 
8 10 Sep
LEC 13:
11 Sep
wednesday is friday
LEC 14:
13 Sep
midsem
9 17 Sep
midsem
20 Sep
midsem
 
10 24 Sep
LEC 15:
27 Sep
LEC 16:
 
11 1 Oct
LEC 17:
4 Oct
LEC 18:
 
12 8 Oct
sembreak
11 Oct
sembreak
 
13 15 Oct
LEC 19:
18 Oct
LEC 20:
 
14 22 Oct
LEC 21:
25 Oct
LEC 22:
26 Oct
saturday is friday
LEC 23:
15 29 Oct
LEC 24:
1 Nov
LEC 25:
 
15 5 Nov
Project presentations
8 Nov
Project presentations
 
15 12 Nov
LEC 26: