![\"\"](\"https:\/\/muthu.co\/wp-content\/uploads\/2018\/06\/download.png\")
<\/a><\/figure><\/div>\n\n\n\nSupport Vector machines are based on the concept of finding the best and the widest plane that divides a set of data. The advantage of using SVM over other classification algorithms like K-Means or Naive Bayes is that SVM finds the decision boundary<\/a> that maximizes the distance from the nearest data points of all the classes. An SVM doesn’t merely find a decision boundary; it finds the most optimal decision boundary. Take a look at the below illustration to understand it better:<\/p>\n\n\n\n In the above illustration, you can see two set of dots, blue and black divided by a single line called the Hyperplane. The margin is the widest road separating the two sets. As marked in the diagram there are few data points which touch the highest margin line, these are called our support vectors (the reason why this algorithm is called Support Vector Machine).<\/p>\n\n\n\n But not all datasets can be divided by a straight line. Take a look at the below dataset:<\/p>\n\n\n\n This is the type we call a non-linear dataset which we usually get for our real world applications. We have to transform these into a form which can be linearly separated. We do that by using functions called kernels.<\/p>\n\n\n\n Transforming Linear data using Linear Kernel:<\/p>\n\n\n\n Transforming data using higher order polynomial or gaussian kernel:<\/p>\n\n\n\n Maths behind finding the hyperplane has been nicely explained here<\/a>, so I will leave the mathematics part and jump right to the implementation.<\/p>\n\n\n\n With the basics in place, first lets try SVM on a random dataset which is linearly separable and understand the various parts of this machine learning algorithm.<\/p>\n\n\n\n Our dataset generated from the above code looks like:<\/p>\n\n\n\n Here “category” is the two groups represented by 0 and 1 for our feature1 and feature2 combination. When plotted we get the below figure, our job using SVM is to find a plane which divides these two datasets.Feeding the above dataset into our SVM :<\/p>\n\n\n\n See how simple it is to import SVM algorithm from sklearn and build our prediction modal. Lets evaluate our modal using the classification report:<\/p>\n\n\n\n The precision is pretty good. Its 100% accurate for our test data. Now, lets find our hyperplane line and the support vectors and plot them too.<\/p>\n\n\n\n In the above plot, you can see the data set being divided by the most optimal line called the hyperplane and also the support vectors touching the decision boundaries.<\/p>\n\n\n\n Now, lets try to implement SVM on a real world dataset. For our analysis we will use the famous Iris dataset<\/a> which consists of 50 samples from each of three species of Iris<\/i> (Iris setosa<\/a><\/i>, Iris virginica<\/a><\/i> and Iris versicolor<\/a><\/i>). Four features<\/a> were measured from each sample: the length and the width of the sepals<\/a> and petals<\/a>, in centimetres. Based on the combination of these four features we will be building an SVM model to distinguish the species from each other.<\/p>\n\n\n\n The dataset looks like:<\/p>\n\n\n\n First attempt : Polynomial Kernel – Degree 8<\/strong><\/p>\n\n\n\n A polynomial kernel is about 87% efficient in classifying the correct flower.<\/p>\n\n\n\n Second Attempt: Gaussian Kernel<\/strong><\/p>\n\n\n\n There isn’t much to change in our program except changing the parameter kernel to ‘rbf’.<\/p>\n\n\n\n Our classification report now looks like :<\/p>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\n# importing scikit learn with make_blobs\nfrom sklearn.datasets.samples_generator import make_blobs\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n\n# Y containing two classes\nX, y = make_blobs(n_samples=50, centers=2, random_state=0, cluster_std=0.40)\n# plotting our dataset \nplt.scatter(X[:, 0], X[:, 1], c=Y, s=50, cmap='spring');\ndf = pd.DataFrame(dict(feature1=X[:,0], feature2=X[:,1], category=y))\n<\/code><\/pre>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\n#split my data set into 80% training and 20% test data\nfrom sklearn.model_selection import train_test_split \nX_train, X_test, y_train, y_test = train_test_split(df[['feature1','feature2']], df['category'], test_size = 0.20)\n\n# #the SVM classification of dataset\nfrom sklearn.svm import SVC \nsvclassifier = SVC(kernel='linear') \nsvclassifier.fit(X_train, y_train)\ny_pred = svclassifier.predict(X_test)<\/code><\/pre>\n\n\n\nfrom sklearn.metrics import classification_report\n#evaluating our model for prediction accuracy\nprint(classification_report(y_test, y_pred))<\/code><\/pre>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\nw = svclassifier.coef_[0]\na = -w[0] \/ w[1]\nxx = np.linspace(-5, 5)\nyy = a * xx - (svclassifier.intercept_[0]) \/ w[1]\n\nb = svclassifier.support_vectors_[0]\nyy_down = a * xx + (b[1] - a * b[0])\nb = svclassifier.support_vectors_[-1]\nyy_up = a * xx + (b[1] - a * b[0])\n\n# plot the line, the points, and the nearest vectors to the plane\nplt.plot(xx, yy, 'k-')\nplt.plot(xx, yy_down, 'k--')\nplt.plot(xx, yy_up, 'k--')\n\nplt.scatter(svclassifier.support_vectors_[:, 0], svclassifier.support_vectors_[:, 1],\n s=80, facecolors='none')<\/code><\/pre>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\nimport numpy as np \nimport matplotlib.pyplot as plt \nimport pandas as pd\n\nurl = \"https:\/\/archive.ics.uci.edu\/ml\/machine-learning-databases\/iris\/iris.data\"\n\n#the imported dataset does not have the required column names so lets add it\ncolnames = ['sepal-length', 'sepal-width', 'petal-length', 'petal-width', 'Class']\nirisdata = pd.read_csv(url, names=colnames)\nX = irisdata.drop('Class', axis=1) #x contains all the features\ny = irisdata['Class'] #contains the categories\n\n#split my data set into 80% training and 20% test data\nfrom sklearn.model_selection import train_test_split \nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.20)\n\n#the SVC algorithm work\nfrom sklearn.svm import SVC \nfrom sklearn.metrics import classification_report, confusion_matrix\nsvclassifier = SVC(kernel='poly', degree=8) \nsvclassifier.fit(X_train, y_train)\ny_pred = svclassifier.predict(X_test)\n\n#evaluating our model for prediction accuracy\ndf_cm = confusion_matrix(y_test, y_pred)\nprint(classification_report(y_test, y_pred))\n\n#seaborn is used for a better looking confusion matrix\nimport seaborn as sn\nsn.heatmap(df_cm, annot=True,annot_kws={\"size\": 16})<\/code><\/pre>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\nsvclassifier = SVC(kernel='rbf')<\/pre>\n\n\n\n