Regression Techniques: Autumn 2018

Course information

• Instructor: Deepayan Sarkar <deepayan@isid.ac.in>
• Syllabus:
  

  Multiple linear regression; partial and multiple correlations; properties of least squares residuals; forward, backward and stepwise regression; different methods for subset selection.

  Violation of linear model assumptions:

  • Lack of fit (linearity): diagnostics and test, Model building.
  • Heteroscedasticity: consequences, diagnostics, tests (including Breusch-Pagan test and White’s test) and efficient estimation.
  • Autocorrelation: consequences, diagnostics, tests (including Durbin-Watson test, Breusch-Godfrey LM test and Durbin’s h-test) and efficient estimation.
  • Collinearity: consequences, diagnostics and strategies (including ridge & shrinkage regression, LASSO, dimension reduction methods).
  • Discordant outlier and influential observations: diagnostics and strategies.

  Robust regression techniques: LAD, LMS and LTS regression (brief exposure).

  Log-Linear models. Introduction to Generalized Linear Models (GLMs), illustration with logit and probit analysis. Linear predictor, link function, canonical link function, deviance. Maximum likelihood estimation using iteratively re-weighted least square algorithm. Goodness of fit test.

  Introduction to nonparametric regression techniques: Kernel regression, local polynomial, knn and weighted knn methods.

  Data analysis and application of the above methods with computer packages.

• Evaluation scheme:
  
  • Midterm examination: 30%
  • Final examination: 50%
  • Assignments / Projects: 20%

Assignments

• Assignment 1 (due 23 August) [solution]
• Assignment 2 (due 16 September) [solution]

Project

Data analysis projects are to be done in groups of two or three. See this page for more details.

Teaching material

• R coding style guides from Google and from Hadley Wickham
• R tutorials (notes)
• Introduction and Review of Linear Regression [handout]
• An Overview of the R Programming Environment [handout]
• Introduction to Non-parametric Regression [handout]
• Unusual and Influential Observations [handout]
• Code demo (30 August)
• Diagnosing Systematic Model Violations [handout]
• Robust Regression [handout]
• Generalized Linear Models [handout]
• Model Selection [handout]
• Collinearity [handout]
• Penalized Regression [handout]