2021-2022 Sem II

This course introduces cloud infrastructure. Students should feel more comfortable with building and deploying their cloud services after having done this course.

Course Information

  • Prerequisites: COL331 or equivalent.

    Note: The course includes programming assignments and thus expects proficiency with systems programming and debugging.

  • Credits: 3-0-2
  • Slot: AB, Mondays and Thursdays 3:30-4:45pm in MS Teams.
  • TAs:
    • Nutesh Sahu: jcs212242 AT csia.iitd.ac.in
    • Soumen Basu: soumen.basu AT cse.iitd.ac.in
    • Abhisek Panda: csz202445 AT cse.iitd.ac.in
  • TA Office hours: TBD
  • Reading material: There is no textbook for the course. Most lectures will link to more reading material.

Grading criteria

  • 30% labs (programming assignments)
  • 20% project
  • 10% assignments
  • 20% minor exam
  • 20% major exam

Supporting systems

  • Lectures will be held in the course Teams channel.
  • Assignments will be regularly released on gradescope.
  • Labs are to be done on Baadal. You will need VPN access to IITD network!
  • Discussions should be done on Piazza.

Acknowledgements

Thanks to Robert T. Morris, MIT and Mythilli Vutukuru, IITB; parts of this course have been inspired by courses made available by them.

Policies

Audit criteria

30% or more marks.

Ethics

We will employ various methods to catch cheating. Cheating in labs/assignments will bring zero in that lab/assignment.

Late policy

  • To help you cope with unexpected emergencies, you can hand in your Labs solutions late, but the total amount of lateness summed over all the lab deadlines must not exceed 72 hours. You can divide up your 72 hours among the labs however you like; you don’t have to ask or tell us. You can only use late hours only for Labs.
  • Assignments can not be submitted late. 1 assignment in the course can be skipped without penalty.

Tentative topics

  • Virtualization: containers, orchestration, hypervisors
  • Recoverability: journaling, snapshotting
  • Fault tolerance: state transfer, replicated state machines
  • Consistency and availability: PACELC theorem
  • Storage Scalability: sharding, consistent hashing
  • Cloud programming: dataflow computation, pub-sub, locking, transactions
  • Light coverage of other topics: cloud economics, public cloud offerings, security

While discussing these topics, we plan to study popular cloud offerings: containers such as docker, orchestration in k8s, key-value stores such as Redis, co-ordination service such as Zookeeper, SQL/NoSQL databases, distributed file systems such as HDFS, pub-sub system Kafka, and dataflow computation in Spark.

Disclaimer: Actual course contents may differ slightly depending on student interest. Reach out to the instructor as soon as possible if there is a particular interest in a topic.

Tentative Schedule

Week Monday Thursday Sunday
1 3 Jan
LEC 1: Introduction
6 Jan
LEC 2: What is scalability? Task DAGs.
Ch.5 of Introduction to Parallel Computing
 
2 10 Jan
LEC 3: Fault-tolerant embarrasingly parallel programs.
MapReduce
13 Jan
LEC 4: Work pool model. Introduce Lab 1.
Celery Optional: Celery at Instagram
 
3 17 Jan
LEC 5: Struggles with Distributed shared memory.
DSM survey
20 Jan
LEC 6: Resilient Distributed Datasets.
Spark
23 Jan
Lab 1 DUE
4 24 Jan
LEC 7: Streaming computation as mini-batches.
Spark Streaming
27 Jan
 
5 31 Jan
3 Feb
6 Feb
Lab 2 DUE
6 7 Feb
10 Feb
 
7 14 Feb
Minors
17 Feb
Minors
20 Feb
Project proposal DUE
8 21 Feb
24 Feb
 
9 28 Feb
Semester break
3 Mar
6 Mar
Lab 3 DUE
10 7 Mar
10 Mar
 
11 14 Mar
17 Mar
 
12 21 Mar
24 Mar
27 Mar
Project DUE
13 28 Mar
Project presentations
31 Mar
Project presentations