Step by Step Sentiment analysis on Twitter data using R with Airtel Tweets: Part – III

After lot of difficulties my 3rd post on this topic in this weekend. In my first post we saw what is sentiment analysis and what are the steps involved in it. In my previous post we saw how to retrieve the tweets and store it in the File step by step. Now we will move on to the step of Sentiment analysis.

Goal: To do sentiment analysis on Airtel Customer support via Twitter in India.

In this Post: We will retrieve the Tweets which are retrieved and stored in the previous post and start doing the analysis. In this post I’m going to use the simple algorithm as used by Jeffrey Breen to determine the scores/moods of the particular brand in twitter.

We will use the opinion lexicon provided by him which is primarily based on Hu and Liu papers. You can visit their site for lot of useful information on sentiment analysis. We can determine the positive and negative words in the tweets, based on which scoring will happen.

Step 1: We will import the CSV file into R using read.csv and you can use the summary to display the summary of the dataframe.

Step 2: We can load the Positive words and Negative words and store it locally and can import using Scan function as given below:


Step 3:

Now we will look at the code for evaluating the Sentiment. This has been taken from http://jeffreybreen.wordpress.com/2011/07/04/twitter-text-mining-r-slides/. Thanks for the source code by Jeffrey.


Step 4:

We will test this sentiment.score function with some sample data.


In this step we have created test and added 3 sentences to it. This contains different words which may be positive or negative. Pass this “Test” to the score.sentiment function with pos_words and neg_words which we have loaded in the previous tests. Now you get the result score from the score.sentiment function against each sentence.

we will also try understand little more about this function and what it does:

a. Two libraries are loaded they are plyr and stringr. Both written by Hadley Wickham one of the great contributor to R. You can also learn more about plyr using this page or tutorial. You can also get more insights on split-apply-combine details here best place to start according to Hadley Wickham. You can think of it on analogy with Map-Reduce algorithm by Google which is used more in terms of Parallelism. stringr makes the string handling easier.

b. Next laply being used. You can learn more on what apply functions do here. In our case we pass on the sentences vector to the laply method. In simple terms this method takes each tweet and pass on to the function along with Positive and negative words and combines the result.

c. Next gsub helps to handle the replacements with the help using gsub(pattern, replacement, x).

d. Then convert the sentence to lowercase

e. Convert the sentences to words using the split methods and retrieve the appropriate scores using score methods.

Step 5: Now we will give the tweetsofaritel from airteltweetdata$text to the sentiment function to retrieve the score.

Step 6: We will see the summary of the scores and its histogram:

The histogram outcome:

It shows the most of the response out of 1499 is negative about airtel.

Disclaimer: Please note that this is only sample data which is analyzed only for the purpose of educational and learning purpose. It’s not to target any brand or influence any brand.

Getting started with TwitteR Package

The intention of this blog post is to give you start on using the TwitteR Package of R. Using this package you can do lot of analysis on social media “Twitter”. I have written an post on analyzing a Cricketer’s Century Tweets and also the need for analyzing tweets already in my blog.

Pre-Requisite tools & Environment:

We are going to explore this completely with Windows 7 and R.

Steps to follow:

Step 1:We need to use the TwitteR package and ROAuth package for accessing the tweets. As per the recommendation from Twitter its always safe to access the tweets via SSL. First we will see the code for the same.

#install the basic packages

install.packages(“ROAuth”)

install.packages(“twitteR”)

#Initiate/Invoke the libraries

library(“ROAuth”)

library(“twitteR”)

#necessary step for Windows to handle the SSL Part

download.file(url=”http://curl.haxx.se/ca/cacert.pem”, destfile=”cacert.pem”)

Step 2: Use the OAuthFactory to setup the Credentials and start accessing data in the following way

cred <- OAuthFactory$new(consumerKey=’azbiz8LbVeA0lBUVh3c6lA‘,

consumerSecret=’Sq5kNMbdYoxNc616urV1Ayi0rKizwePRg2tDkIUEk‘,

requestURL=’https://api.twitter.com/oauth/request_token&#8217;,

accessURL=’http://api.twitter.com/oauth/access_token&#8217;,

authURL=’http://api.twitter.com/oauth/authorize&#8217;)


After this you can notice that “handshakeComplete” is FALSE. We need to complete the handshake to get access to the TwitterAPI and its data.

Step 3: Create a handshake with twitter, for which you will get a message like the following:

To enable the connection, please direct your web browser to:
http://api.twitter.com/oauth/authorize?oauth_token=ZS2khFL8LZmd4XZ92yeCjcchX08E80g3uzUucv6ds
When complete, record the PIN given to you and provide it here: install.packages("ROAuth")
Error: Unauthorized

Once you naviage to the URL you will get a PIN which you should type in the R Console. Now you can see that we have enterered the PIN from the browser after authorizing the Application.

You can also realize that now the “handshakeComplete” has become TRUE.

Step 4: Verify the status of OAuth authentication using the following command and it should return TRUE.

registerTwitterOAuth(cred)


Step 5: Now the next step is to start accessing the data using TwitterAPI. Let’s try to get started with accessing the User Information.

userInfo<-getUser(“seesiva”, cainfo=”cacert.pem”)

You need to make sure that you also pass the cainfo otherwise you will get an SSL Error.


Hope now we understand the steps required for accessing the Twitter data using the TwitteR Package. In this example we had shown the various attributes of the User Object retrieval. In the next post we will try to analyze some data.

Similar posts for your reference:

http://yourwhatyourepeatedlydo.blogspot.in/2013/04/downloading-twitter-data-using-r.html

http://davetang.org/muse/2013/04/06/using-the-r_twitter-package/

Twitter Analytics & Business impact (The 140 Character thing)

Hopefully everyone who reads this blog would have a twitter account. Not a necessary evil, huh?

Have you ever wondered whats going on about you, your business, the technology you’re in the twitter world. Welcome to Twitter Analytics.  Though twitter API has given as much of exposure to understand these woes better.

Why should I worry about twitter when I have plenty of things to worry about my business:

a. You can find your customers talking about you bad and good which would like to know for dynamically align your business strategy

b. You can acquired customers, you will come to know about customer problems

c. You can predict the online presence and your brand value with the help of your followers and their growth

d. Its real-time and can be viral

e. Above all its gets you closer to the customer

In the following posts we will see more how we can do analytics on the tweets.