INDIAN STATISTICAL INSTITUTE
(Headquarters at Kolkata)
203 B. To Road, Kolkata 700 108
Introduction to Stochastic Processes
B. Stat. IInd Year
Academic Year 2020  2021: Semester II
Instructor: Antar Bandyopadhyay
Email: antar (at) isid (dot) ac (dot) in
Office:
 Room # 208 on the First Floor of the Faculty Block (when in Delhi); and
 Room # 3.4 on the Third Floor of the A. N. Kolmogorov Bhavan
(when in Kolkata).
Teaching Assistant (TA): Arijit Pyne
Email: arijitpynestat (at) gmail (dot) com
Weekly Class Time:
 Lecture Hours: Tuesday and Thursday 03:00 PM  05:00 PM.
Lecture Hall: Online Google Classroom and
online class via Zoom Meetings
Course Duration (including examinations): January 25  May 28, 2021
[Total of 18 Weeks = 14 Weeks of Classes + 1 week of Midterm + 1 week of Study Leave + 2 weeks of Final Examination Weeks].
Midterm Examination:
Date: TBA (during March 15  19, 2021)
Time: TBA
Venue: TBA
Final Examination:
Date: TBA (during May 17  28, 2021)
Time: TBA
Venue: TBA
Course Outline:

Discrete Markov chains with countable state space,
Examples including two state chain, SSRW, random walk, gambler's ruin,
birth and death chain, renewal chain, Ehrenfest chain,
card shuffling and branching processes.

Classification of states, recurrence and transience; absorbing states,
irreducibility,
decomposition of state space into irreducible classes, examples.

Absorbing chains, absorption probabilities and mean absorption times. Fundamental
matrix.

Stationary distributions, limit theorems, positive and null recurrence.
Reversible chains.

Definition of periodicity, periodic and aperiodic chains.
Cyclic decomposition of a periodic chain.

Limit theorems for aperiodic irreducible chains. Ratio limit theorems.

Concept of mixing and relaxation of a finite state irreducible chain. Cover time and
strong uniform time (if time permits).

Exact sampling, relation with strong uniform time. Markov chain Monte Carlo (MCMC)
and convergence issues (if time permits).
Poisson process: basic properties,
conditional distributions of arrival times as order statistics, applications.
Nonhomogeneous and compound Poisson processes. Simple birth and death
processes.

Introduction to general continuous time but discrete state space Markov processes.
Kolmogorov's forward and backward equations.
References:
 W. Feller:
Introduction to the Theory of Probability and its Applications: Volume I.
 P. G. Hoel, S. C. Port and C. J. Stone:
Introduction to Stochastic Processes.
 S. M. Ross:
Stochastic Processes.
 D. Aldous and J. Fill:
Reversible Markov Chains and Random Walks on Graphs
(available online at
https://www.stat.berkeley.edu/~aldous/RWG/book.html).
 J. G. Kemeny, J. L. Snell and A. W. Knapp:
Finite Markov Chains.
 C. P. Robert and G. Casella:
Monte Carlo Statistical Methods.
Grading Policy:
 Assignments: 20% of the total credit.
 Midterm: 30% of the total credit
 Final Exam: 50% of the total credit
Assignment Policies:
 There will be a total of 14 sets of homework assignments each
carrying a total of 20 points. The average of 10 best assignment scores will be taken for the final grading.
 The assignments will be given every Thursday,
starting from January 28, 2021. Each
assignment will be due on the following Thursday. For
example the first assignment is due on Thursday, February 04, 2021.
 Each assignment will be based on the course materials which will be
covered in the class in the week of the assignment.
 Late submission of an assignment will NOT BE ACCEPTED.
 Graded assignments will be returned individually.
Exam Policies:
 Both the Midterm and the Final Examination will be
open notes examinations. That means, students will be allowed
to bring his/her own hand written notes, study materials, list of
theorems etc. But no printed or photocopied materials will be allowed.
 Any unfair means used by any students in the examinations will be dealt with the strictest
possible measures, as per the Institute rules. In particular, if any student is found to be
using any kind of unfair practice during any of the examinations (including the quizzes) then
he/she will be awarded ZERO in that examination.
Regrading Policy:
 Regrading of the assignments or exams will only be undertaken in cases where, you believe there has been a
genuine error or misunderstanding. Please note that our primary aim in grading is consistency,
so that all students are treated the same; for this reason, we will not adjust the score of one student
on an issue of partial credit, unless the score allocated clearly deviates from the grading policy
we adopted for that problem.
 If you wish to request a regrading of a homework or exam, you must return it to the instructor
with a written note on a separate piece of paper explaining the problem.
 The entire assignment or the exam may be regraded, so be sure to check the solutions to ensure that your
overall score will go up after regrading.
 All such requests must be received within one week from the date on which the homework or exam was made
available for return.
Last modified February 02, 2021.