Support Vector Machines (SVM) Intro

Back to Index

---

Objective

Learn SVM

Essentials Reading

Understanding Classifications

Read the basics of classifications

Basics of SVM

• Understanding Support Vector Machine(SVM)
• Support Vector Machines for Machine Learning
• support vector machine (SVM) in layman’s terms
• A good animation of kernel trick

Implementing SVM in Scikit-Learn

• SVM in Scikit Learn documentation
• Support Vector Machines with Scikit-learn

Extra Reading

• Ch 9 of book Introduction to Statistical Learning offers good overview of SVM

Checklist

Check your knowledge:

• What is a maximal-margin classifier?
• What is slack (C) ?
• What are the strengths and weaknesses of SVM?
• Do we need to scale/normalize data before using SVM?
• What is a kernel trick?
• Q1: Which of the following statements are true for SVM (can be more than one)
  • SVM exhibits high variance
  • SVM exhibits high bias
  • SVM can be used for classification
  • SVM can be used for clustering
  • SVM can only do linear separation
  • SVM required that we scale data

Exercises

Difficulty Level

★☆☆ - Easy
★★☆ - Medium
★★★ - Challenging
★★★★ - Bonus

EX-1: Practice with synthetic data

Use Scikit’s make_blobs or make_classification to generate some sample data.

Try to separate them using SVM

EX-2: College Admission Data

Load college admission data

It will look like this:

    admit  gre   gpa  rank
0       0  380  3.61     3
1       1  660  3.67     3
2       1  800  4.00     1
3       0  640  3.19     4
4       0  520  2.93     4
...

• Use input features: gre, gpa, rank, and predict output: admit
• Use Logistic Regression to predict.
• Create a confusion matrix
• What is the accuracy of the model

EX-3 - BONUS Lab

• Choose a classification dataset from UCI data repository
• For example, consider US Census Income data
• Run SVM on the data of choice

More Exercises

---

Index