Linear Regression Intro

Back to Index

---

Objective

Learn Linear Regression

Essential Reading

Basics of Linear Regression

• “Statistics PL14 - Simple Linear Regression” playlist - a video series by Brandon Foltz offers pretty good coverage of Linear Regression. Start with the following videos
  • Statistics 101: Linear Regression, The Very Basics
  • Statistics 101: Linear Regression, Algebra, Equations, and Patterns
  • Statistics 101: Linear Regression, The Least Squares Method
  • Statistics 101: Linear Regression, Fit and Coefficient of Determination
  • Statistics 101: Linear Regression, Standardized Regression
  • Statistics 101: Linear Regression, Understanding Model Error

Understanding Errors and Residuals

• Read Errors an Residuals section in Regressions

Implementing Linear Regression in Scikit-Learn

• Linear Regression for Machine Learning
• Linear Regression wit Scikit-Learn - a tutorial

Extra Reading

• Continue watching the rest of the videos in “Statistics PL14 - Simple Linear Regression” playlist
• Ch 3 - Linear Regression (pages 59 - 126) from Introduction to Statistical Learning book, covers some good theory and math behind LR

Checklist

• What do the following mean?
  • coefficients
  • slope
  • intercept
• Understand different error metrics
  • Sum of Squared Error (SSE)
  • Mean square error (MSE)
  • Mean absolute error (MAE)
  • Root mean squared error (RMSE)
• Regression evaluation metrics
  • r (Person’s coefficient)
  • r² (Coefficient of determination)
• Generating synthetic dataset for regression (sklearn.datasets.make_regression)
• Familiar with sklearn.linear_model.LinearRegression

Exercises

Difficulty Level

★☆☆ - Easy
★★☆ - Medium
★★★ - Challenging
★★★★ - Bonus

Ex-1 - Practice regression with synthetic data (★☆☆)

Here we will use Scikit’s make_regression to generate some data and fit linear regression.

Start with this notebook : lr-1__intro

And work through it

Ex-2 - Billing and tipping data (★☆☆)

Start with this notebook: lr-2__tips

Complete the TODO items

Ex-3 - House sales data (★★☆)

• Data: house-sales-full.csv
• Label is : SalePrice
• Input features : Start with Bedrooms, Bathrooms, SqFtTotLiving, SqFtLot
• Run linear regression
• What is the R² you are getting?
• Can you reason why the R² is low?
• Add more features to input and see if you can improve the score
• What is the maximum R² you can attain? With what features?

---

Index