Markov Chains
Fall 2012
Instructor: Antar Bandyopadhyay
(Email: antar (at) isid (dot) ac (dot) in
Office: 208 Faculty Building).
Class Time: Tue Fri 11:00 - 13:00 in Class Room 23.
Instructor's Office Hours: Tue Fri 13:00 - 14:30
Course Duration: September 3 - October 5, 2012 and October 29 - November 2, 2012 (total of 5 weeks)
Midterm Examination: September 19, 2012, Time: 10:00 - 13:00 hours, Place: Conference Hall, Administrative Block
Final Examination: November 14, 2012, Time: 10:00 - 12:00 hours, Place: Conference Hall, Administrative Block
Course Outline:
- Discrete Markov chains with countable state space, Examples including two state chain, SSRW, random walk, gambler's ruin, birth and death chain,
renewal chain, Ehrenfest chain, card shuffling and branching processes.
- Classification of states, recurrence and transience; absorbing states, irreducibility, decomposition of state space into irreducible classes,
examples.
- Stationary distributions, limit theorems, positive and null recurrence.
- Definition of periodicity, aperiodic chains.
- Limit theorems for aperiodic irreducible chains.
- Study of random walks, birth-and-death chains and branching processes.
References:
- Introduction to Stochastic Processes by Paul G. Hoel, Sidney C. Port and Charles J. Stone;
- An Introduction to Probability Theory and Its Applications by William Feller;
- Probability An Introduction by Geoffrey Grimmett and Dominic Welsh;
- Probability: Theory and Examples by Rick Durrett (ONLY Chapter 5).
Prerequisites:
- Real Analysis (at the level of Principles of Mathematical Analysis by W. Rudin).
- Liner Algebra (at the level of Finite Dimensional Vector Spaces by P. R. Halmos).
Grading Policy:
- Assignments: 20% of the total credit.
- Midterm Exam: 20% of the total credit.
- Final Exam: 60% of the total credit.
Note:In the course our part will have 25% of credit.
Assignment Policies:
- There will be a total of 5 sets of homework assignments each
carrying a total of 20 points. 4 best assignment scores
will be taken for the final grading.
- The assignments will be given in class on every Tuesday starting from
September 11, 2012
and they will be due in class on the following Tuesday. Each assignment
set will be
based on course materials covered in the lectures in the past week.
For example, the assignment given on September 11, 2012 (Tuesday)
will be due on September 18, 2012 (Tuesday).
- Late submission of an assignment will NOT be accepted. If you
can not submit an assignment on time, don't worry about it, and try to do
well in the others. It will not count in your final grade since you
have one extra assignment anyway.
- Graded assignments will be returned in the class on the
Friday following the due date. For example, the assignment which is due on September 18, 2012 (Tuesday)
will be returned after grading on September 21, 2012 (Friday).
- Click here
for downloading the assignments.
Exam Policies:
- Each examination will be
an open note examination. That means, students are allowed
to bring his/her own hand written notes, study materials, list of
theorems etc.
Last modified November 3, 2012.