INDIAN STATISTICAL INSTITUTE

(Headquarters at Kolkata)

203 B. To Road, Kolkata 700 108

Introduction to Stochastic Processes

B. Stat. IInd Year

Academic Year 2020 - 2021: Semester II

Instructor: Antar Bandyopadhyay
Email: antar (at) isid (dot) ac (dot) in
Office:

• Room # 208 on the First Floor of the Faculty Block (when in Delhi); and
• Room # 3.4 on the Third Floor of the A. N. Kolmogorov Bhavan (when in Kolkata).

Teaching Assistant (TA): Arijit Pyne
Email: arijitpynestat (at) gmail (dot) com

Weekly Class Time:

• Lecture Hours: Tuesday and Thursday 03:00 PM - 05:00 PM.

Lecture Hall: Online Google Classroom and online class via Zoom Meetings

Course Duration (including examinations): January 25 - May 28, 2021
[Total of 18 Weeks = 14 Weeks of Classes + 1 week of Midterm + 1 week of Study Leave + 2 weeks of Final Examination Weeks].

Midterm Examination:
Date: TBA (during March 15 - 19, 2021) Time: TBA
Venue: TBA

Final Examination:
Date: TBA (during May 17 - 28, 2021)
Time: TBA
Venue: TBA

Course Outline:

• Discrete Markov chains with countable state space, Examples including two state chain, SSRW, random walk, gambler's ruin, birth and death chain, renewal chain, Ehrenfest chain, card shuffling and branching processes.

• Classification of states, recurrence and transience; absorbing states, irreducibility, decomposition of state space into irreducible classes, examples.

• Absorbing chains, absorption probabilities and mean absorption times. Fundamental matrix.

• Stationary distributions, limit theorems, positive and null recurrence. Reversible chains.

• Definition of periodicity, periodic and aperiodic chains. Cyclic decomposition of a periodic chain.

• Limit theorems for aperiodic irreducible chains. Ratio limit theorems.

• Concept of mixing and relaxation of a finite state irreducible chain. Cover time and strong uniform time (if time permits).

• Exact sampling, relation with strong uniform time. Markov chain Monte Carlo (MCMC) and convergence issues (if time permits).

• Poisson process: basic properties, conditional distributions of arrival times as order statistics, applications. Non-homogeneous and compound Poisson processes. Simple birth and death processes.

• Introduction to general continuous time but discrete state space Markov processes. Kolmogorov's forward and backward equations.

References:

• W. Feller: Introduction to the Theory of Probability and its Applications: Volume I.
• P. G. Hoel, S. C. Port and C. J. Stone: Introduction to Stochastic Processes.
• S. M. Ross: Stochastic Processes.
• D. Aldous and J. Fill: Reversible Markov Chains and Random Walks on Graphs
  (available online at https://www.stat.berkeley.edu/~aldous/RWG/book.html).
• J. G. Kemeny, J. L. Snell and A. W. Knapp: Finite Markov Chains.
• C. P. Robert and G. Casella: Monte Carlo Statistical Methods.

Grading Policy:

• Assignments: 20% of the total credit.
• Midterm: 30% of the total credit
• Final Exam: 50% of the total credit

Assignment Policies:

• There will be a total of 14 sets of homework assignments each carrying a total of 20 points. The average of 10 best assignment scores will be taken for the final grading.
• The assignments will be given every Thursday, starting from January 28, 2021. Each assignment will be due on the following Thursday. For example the first assignment is due on Thursday, February 04, 2021.
• Each assignment will be based on the course materials which will be covered in the class in the week of the assignment.
• Late submission of an assignment will NOT BE ACCEPTED.
• Graded assignments will be returned individually.

Exam Policies:

• Both the Midterm and the Final Examination will be open notes examinations. That means, students will be allowed to bring his/her own hand written notes, study materials, list of theorems etc. But no printed or photocopied materials will be allowed.
• Any unfair means used by any students in the examinations will be dealt with the strictest possible measures, as per the Institute rules. In particular, if any student is found to be using any kind of unfair practice during any of the examinations (including the quizzes) then he/she will be awarded ZERO in that examination.

Regrading Policy:

• Regrading of the assignments or exams will only be undertaken in cases where, you believe there has been a genuine error or misunderstanding. Please note that our primary aim in grading is consistency, so that all students are treated the same; for this reason, we will not adjust the score of one student on an issue of partial credit, unless the score allocated clearly deviates from the grading policy we adopted for that problem.
• If you wish to request a regrading of a homework or exam, you must return it to the instructor with a written note on a separate piece of paper explaining the problem.
• The entire assignment or the exam may be regraded, so be sure to check the solutions to ensure that your overall score will go up after regrading.
• All such requests must be received within one week from the date on which the homework or exam was made available for return.

Last modified February 02, 2021.