SVM Implementation Step by Step with R: Ice-cream sales prediction

In the previous blog post we saw till the steps of collecting data from the CSV file to segregate the 30% of data for test and remaining 70% of the data for training. Now in this blog we will continue to build the SVM model and arrive at the confusion matrix which will end this series on SVM.

  1. Let use the SVM model using the model<-svm(SalesRating ~ .,data=dataset)
  2. The above statement will be working correctly only if you have loaded e1701 library. We are specifying SalesRating because it’s the class which will be used by SVM for learning.

  3. Now you can see that the model has printed with SVM Type as C-Classification and Kernel used is Radial basis function with a cost 1 and Gamma of 0.5. The support vectors is 40
  4. The summary function will provide you the classification and Levels information also as given in the below picture

  5. Now we will plot the Model against the data so that we can understand the relationship on the sales vs temperature in the city as given below using the command plot(model,dataset):

  6. Now the next step is to really do the prediction, for the purpose predicting we need to classify the data Training Data Set and Test Data Set, which we have already done in our previous blog.
  7. Now using the SVM we will train the system using SVM using the commands given in the screen

  8. The plot output looks like this on the trainingset data:

  9. Now we will move on to prediction using the model which has been trained using the trainingset of data. We will use the prediction<-predict(model,testset[,-1]) for prediction with testset excluding the class column SalesRating.
  10. We would like generate the confusion matrix using the table statement with the following command tab<-table(pred=prediction,true=testset[,1])
  11. When you print the tab you will get the confusion matrix like the following:

  12. The confusion matrix looks like the below :

  13. Now we will find the accuracy of the our prediction the Formula for Accuracy is

  14. Applying the above formula the accuracy is 89.79%

In the future post we will try to explore the scenario of multi-class classification.

Additional References:

http://www2.cs.uregina.ca/~dbd/cs831/notes/confusion_matrix/confusion_matrix.html

http://en.wikipedia.org/wiki/Confusion_matrix

http://en.wikibooks.org/wiki/Data_Mining_Algorithms_In_R/Classification/SVM

Advertisements

SVM: Support Vector Machines and Multi Class Classification

Most of the Classification examples out in the internet talks about binary classification. Also we must understand the learning also applies widely on multi class classification. Good exampleswould be Risk of High/Medium/Low.

According to Wikipedia in machine learning multiclass or multinomial classification is the problem of classifying instances into more than two classes.

Problem:

We would need to test the hypothesis of a patient being in a risk of Heart Attack.

Data Available:

Various set of attributes with pre-defined label of risk of High/Medium/Low as training set of 500 records.

Before we get into the solution we need to understand little bit of SVM.

History of SVM:

Invented by            : Vladimir N Vapnik

Current Standard Proposed by    : Vapnik and Corinna Cortes

Year                 : 1995

Before we even start to understand what is SVM, we need to understand what is Hyperplane. Hyper plane is a concept in geometry if we remember our good school days. We can recollect it well. It is an n-dimensional space. To understand more with examples on hyper planes look at this link.

Similar to any other machine learning techniques, SVMs take some data to start with that’s already classified (the training set), and tries to predict a set of unclassified data (the testing set).

Good URL to know more:

http://www.reddit.com/r/MachineLearning/comments/15zrpp/please_explain_support_vector_machines_svm_like_i/?utm_source=twitterfeed&utm_medium=twitter (I like the most).

http://www.yaksis.com/posts/why-use-svm.html

http://www.cis.upenn.edu/~crammer/code-index.html

We will explore on SVM with a simple example on multi-class classification in my next post.