Facebook Data Mining using R

In this tutorial, we will see how to extract and analyze facebook data using R. Facebook has crossed more than 1 billion active users.  Facebook has gathered the most extensive data set ever about behavior of human. In R, we can extract data from Facebook and later analyze it. Social media mining is one of the most interesting piece in data science. You can analyze sentiments of an important event by pulling information about the event from Facebook and get insights from data in R.
Extract Facebook Data using R
Step by Step Guide : Extract Data from Facebook

Step I : Facebook Developer Registration

Go to https://developers.facebook.com and register yourself by clicking on Get Started button at the top right of page (See the snapshot below). After it would open a form for registration which you need to fill it to get yourself registered.
Facebook  Developer Registration



Step II : Add a new App

Once you are done with registration as shown in step 1, you need to click on My Apps button (check out the snapshot below). Then select Add a New App from the drop down.

Facebook : My Apps

Then you need to write Display Name of App ID (Type any  name) and select drop down option in Category (Choose Education). press Create App ID button.
Create a new App

Step 3 : Get App ID and App Secret

In this step, we need to note down our App ID and App Secret (Refer the screenshot below).
Fb App ID and App Secret

Step 4 : OAuth Settings

  1. On the left hand side menu, click on Add Product Button
  2. Click on Facebook Login link
  3. Under Settings, make sure YES is selected in Client OAuth Login
  4. Type http://localhost:1410/ in Valid OAuth redirect URIs box
  5. Click on Save Changes button

OAuth redirect URIs

If you don't put information correctly, you would get the following error -
Can't Load URL: The domain of this URL isn't included in the app's domains. To be able to load this URL, add all domains and subdomains of your app to the App Domains field in your app settings. 
Step 5 :  Write R Script

1. Install required packages

Go to R and install Rfacebook and RCurl packages. Run the following code to install them.
install.packages("Rfacebook")
install.packages("RCurl")
The package Rfacebook lets you to access Facebook App via R.

2. Load desired packages

In this step, we will load the above installed packages.
library(Rfacebook)
library(RCurl)
3. Paste your app id and app secret below 
fb_oauth <- fbOAuth(app_id="183xxxxxxxx3748", app_secret="7bfxxxxxxxxcf0",extended_permissions = TRUE)
Press ENTER in R Console or CTRL+ENTER in R Studio.

It would return the following message -
Copy and paste into Site URL on Facebook App Settings: http://localhost:1410/ 
When done, press any key to continue...
Waiting for authentication in browser...
Press Esc/Ctrl + C to abort

Authentication in Browser

Authentication Status

4. Check your profile account information
me <- getUsers("me",token=fb_oauth, private_info=TRUE)
me$name
[1] "Deepanshu Bhalla"

Fix : Error

Are you getting the error below?
Error in callAPI(query, token) :  An active access token must be used to query information about the current user.
Recently Facebook has made changes in the API which causes error in functions of Rfacebook package. See the method below to correct it.

Step 1 : Run the following program
fbOAuth <- function(app_id, app_secret, extended_permissions=FALSE, legacy_permissions=FALSE, scope=NULL)
{
  ## getting callback URL
  full_url <- oauth_callback()
  full_url <- gsub("(.*localhost:[0-9]{1,5}/).*", x=full_url, replacement="\\1")
  message <- paste("Copy and paste into Site URL on Facebook App Settings:",
                   full_url, "\nWhen done, press any key to continue...")
  ## prompting user to introduce callback URL in app page
  invisible(readline(message))
  ## a simplified version of the example in httr package
  facebook <- oauth_endpoint(
    authorize = "https://www.facebook.com/dialog/oauth",
    access = "https://graph.facebook.com/oauth/access_token")
  myapp <- oauth_app("facebook", app_id, app_secret)
  if (is.null(scope)) {
    if (extended_permissions==TRUE){
      scope <- c("user_birthday", "user_hometown", "user_location", "user_relationships",
                 "publish_actions","user_status","user_likes")
    }
    else { scope <- c("public_profile", "user_friends")}
 
    if (legacy_permissions==TRUE) {
      scope <- c(scope, "read_stream")
    }
  }

  if (packageVersion('httr') < "1.2"){
    stop("Rfacebook requires httr version 1.2.0 or greater")
  }

  ## with early httr versions
  if (packageVersion('httr') <= "0.2"){
    facebook_token <- oauth2.0_token(facebook, myapp,
                                     scope=scope)
    fb_oauth <- sign_oauth2.0(facebook_token$access_token)
    if (GET("https://graph.facebook.com/me", config=fb_oauth)$status==200){
      message("Authentication successful.")
    }
  }

  ## less early httr versions
  if (packageVersion('httr') > "0.2" & packageVersion('httr') <= "0.6.1"){
    fb_oauth <- oauth2.0_token(facebook, myapp,
                               scope=scope, cache=FALSE)
    if (GET("https://graph.facebook.com/me", config(token=fb_oauth))$status==200){
      message("Authentication successful.")
    }
  }

  ## httr version from 0.6 to 1.1
  if (packageVersion('httr') > "0.6.1" & packageVersion('httr') < "1.2"){
    Sys.setenv("HTTR_SERVER_PORT" = "1410/")
    fb_oauth <- oauth2.0_token(facebook, myapp,
                               scope=scope, cache=FALSE)
    if (GET("https://graph.facebook.com/me", config(token=fb_oauth))$status==200){
      message("Authentication successful.")
    }
  }

  ## httr version after 1.2
  if (packageVersion('httr') >= "1.2"){
    fb_oauth <- oauth2.0_token(facebook, myapp,
                               scope=scope, cache=FALSE)
    if (GET("https://graph.facebook.com/me", config(token=fb_oauth))$status==200){
      message("Authentication successful.")
    }
  }

  ## identifying API version of token
  error <- tryCatch(callAPI('https://graph.facebook.com/pablobarbera', fb_oauth),
                    error = function(e) e)
  if (inherits(error, 'error')){
    class(fb_oauth)[4] <- 'v2'
  }
  if (!inherits(error, 'error')){
    class(fb_oauth)[4] <- 'v1'
  }

  return(fb_oauth)
}

Step 2 :  Run fbOAuth function again

Make sure you put your own app_id and app_secret number before using the code below
fb_oauth <- fbOAuth(app_id="183385******33748", app_secret="7bf18f8********4cf7def77cf0",extended_permissions = TRUE)

Now, getUsers() function will work.


5. List of all the pages you have liked

Suppose you want to see all the pages you have liked in the past.
likes = getLikes(user="me", token = fb_oauth)
sample(likes$names, 10)
The sample() function is used to list some 10 random pages you have liked.

 [1] "The Hindu"                  "ADGPI - Indian Army"        "Brain Humor"            
 [4] "Jokes Corner"               "The New York Times"         "Oye! Extra Pen Hai?"    
 [7] "So You Think You Can Dance" "Shankar Tucker"             "Rihanna"                
[10] "Lindsey Stirling"


6. Update Facebook Status from R

You can also update status in Facebook via R.
updateStatus("this is just a test", token=fb_oauth)

7. Search Pages that contain a particular keyword
pages <- searchPages( string="trump", token=fb_oauth, n=200)
 In the above code, we are telling R to search all the pages that contain 'trump' as keyword. The n= 200 refers to the number of pages to return.

It returns 16 variables. See the list of variables -

[1] "id"                  "about"               "category"        
 [4] "description"         "general_info"        "likes"            
 [7] "link"                "city"                "state"            
[10] "country"             "latitude"            "longitude"        
[13] "name"                "talking_about_count" "username"        
[16] "website"
head(pages$name)
[1] "Donald J. Trump"                 "Ivanka Trump"                
[3] "President Donald Trump Fan Club" "President Donald J. Trump"    
[5] "Donald Trump Is My President"    "Donald Trump For President"  


8. Extract list of posts from a Facebook page

See the status posted by BBC News. The facebook page name of BBC News is bbcnews.
page <- getPage(page="bbcnews", token=fb_oauth, n=200) 
Posts Details
The above image is truncated. It returns in total 11 variables. See the variables' list -

 [1] "from_id"        "from_name"      "message"        "created_time"
 [5] "type"           "link"           "id"             "story"      
 [9] "likes_count"    "comments_count" "shares_count

9. Get all the posts from a particular date

You can also put the beginning and end date of the posts you wanted to extract.
page <- getPage("bbcnews", token=fb_oauth, n=100,
since='2016/06/01', until='2017/03/20')

10. Which of these posts got maximum likes?

To know the most popular BBCNews post, you can submit the following line of code.
summary = page[which.max(page$likes_count),]
summary$message
[1] "Could circular runways take off? (via BBC World Hacks)"

11. Which of these posts got maximum comments?

Some posts are not so popular in terms of likes but they fetch max comments. It might be because they are controversial.
summary1 = page[which.max(page$comments_count),]
"When Angela Merkel met Donald J. Trump, did her reactions speak louder than words?

12. Which post was shared the most?
summary2 = page[which.max(page$shares_count),]
"Islam will be the world's largest religion by 2070, new research suggests."

13. Extract a list of users who liked the maximum liked posts

In terms of marketing or growth of a website, it is very important to know about the users who liked a certain post.
post <- getPost(summary$id[1], token=fb_oauth, comments = FALSE, n.likes=2000)
To view the list of people:
likes <- post$likes
head(likes)
Result - 
from_name           from_id
Tommy Johnson 10154527932013108
Mirtunjay Raj   399490251425210
Sony Joseph   142559101272027

Note - I have edited the IDs to maintain privacy


14. Extract FB comments on a specific post

To know what users think about a post, it is important to analyze their comments.
post <- getPost(page$id[1], token=fb_oauth, n.comments=1000, likes=FALSE)
comments <- post$comments
fix(comments)

15. What is the comment that got the most likes?
comments[which.max(comments$likes_count),]
16. What are the most common first names in the user list?
head(sort(table(users$first_name), dec=TRUE), n=3)
  David   John Daniel
    14     13     10

17. Extract Reactions for most recent post

Facebook has more than a like button. Last year, it launched emoji (emoticons). If a post got 1k likes, it does not mean everyone really loves the comment. The reaction can be happy, sad or angry.
post <- getReactions(post=page$id[1], token=fb_oauth)
love_count = 60, haha_count = 286, wow_count = 62, sad_count = 169, angry_count = 532

18. Get Posts of a particular group

First, searchGroup() function searches id of a group from which you want to pull out posts. Later, the group ID is used as a input value in getGroup() function.
# Extract posts from Machine Learning Facebook group
ids <- searchGroup(name="machinelearningforum", token=fb_oauth)
group <- getGroup(group_id=ids[1,]$id, token=fb_oauth, n=25)
In case, searchGroup() function could not find group id. You can search it on lookup-id website.

End Notes

Text Mining (Social) has gained a lot of interest in a last couple of years. Every company has started analyzing customers' opinion about their products and what customers talk about the company in social media world. It helps marketing team to define marketing strategies and development team to modify the upcoming products based on customer feedback.

R Tutorials : 75 Free R Tutorials

About Author:

Deepanshu founded ListenData with a simple objective - Make analytics easy to understand and follow. He has over 7 years of experience in data science and predictive modeling. During his tenure, he has worked with global clients in various domains like banking, Telecom, HR and Health Insurance.

While I love having friends who agree, I only learn from those who don't.

Let's Get Connected: Email | LinkedIn

Get Free Email Updates :
*Please confirm your email address by clicking on the link sent to your Email*
Related Posts:
61 Responses to "Facebook Data Mining using R"
  1. this step bombed:

    me <- getUsers("me", token=fb_Oauth, private_info=TRUE)

    Error in callAPI(query, token) :
    An active access token must be used to query information about the current user.

    ReplyDelete
    Replies
    1. I have added a method in the post to fix this error.

      Delete
    2. This comment has been removed by the author.

      Delete
    3. Yes, you need to run both the steps - Step 1 and Step 2. Run one by one. Make sure you change app_id and app_secret number before running step2 code.

      Delete
  2. me <- getUsers("me", token=fb_Oauth, private_info=TRUE)

    Error in callAPI(query, token) :
    An active access token must be used to query information about the current user.

    Reply

    ReplyDelete
    Replies
    1. facing the same issue.
      any fix for this pls?????

      Delete
    2. @Deepanshu....thank you for the original post and then for the fix as well. Will test it out and let you know. Looking forward to more learning. Thanks again.

      Delete
    3. @ Deepanshu.
      I just run the fix. It works. awesome . thanks a ton!

      Delete
  3. @ Step 7 :
    I am getting this error:
    In vect[notnulls] <- unlist(lapply(lst, function(x) x[[field]])) :
    number of items to replace is not a multiple of replacement length

    ReplyDelete
    Replies
    1. It's a warning. It returns information about public pages that match the keyword. It seems to be some issues with handling missing data in this function.

      Delete
    2. I also face the same error admin please help out to resolve

      Delete
    3. iam also facing same issue with searchpages method.due to this iam unable to proceed furthur.can u plz help me

      Delete
  4. Thanks very much for the fix you've published!

    ReplyDelete
  5. it's a very useful tutorial! it helped me a lot, thank you very much

    ReplyDelete
  6. Amazing content! It was very helpful, especially your error handling

    ReplyDelete
  7. Great content..!

    ReplyDelete
  8. when i do step 8, the only output i get is
    "25 posts 50 posts 75 posts 100 posts 125 posts 150 posts 175 posts 200 posts" instead of the list. help me sir

    ReplyDelete
  9. Any method for deriving a list of users who have liked a particular facebook page?

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
  10. we get only name and id how can we get emails ?

    ReplyDelete
  11. Companies can use Facebook to communicate with the right audience, the target goal and have a greater chance of success. http://www.thesocialguys.co.uk

    ReplyDelete
  12. I seem to be stuck and running in loops
    I am trying to scrape a list of facebook posts, NOT FROM A PARTICULAR GROUP, and extract basic information like SHares, Likes and date of activity. It seems that the Rfacebook packages may be limited is that correct?

    ReplyDelete
  13. I have this problem, in fact in your step 13, is the same error, can you help me? thanks


    >page <- getPage(page="luismiguelofficial", token=fb_oauth, n=200)
    25 posts 50 posts 75 posts 100 posts 125 posts 150 posts 175 posts 200 posts >

    > summary = page[which.max(page$likes_count),]

    > post <- getPost(post=summary$id[1], token=fb_oauth, n=100, comments =TRUE)

    Error in while (n.l < n.likes & length(content$data) > 0 & !is.null(url <- content$paging$`next`)) { :
    argument is of length zero

    ReplyDelete
  14. how do you extract facebook status using rstudio

    ReplyDelete
  15. > ids <- searchGroup(name="machinelearningforum", token=fb_oauth)

    Error in data.frame(name = unlist(lapply(content$data, "[[", "name")), :
    arguments imply differing number of rows: 1, 0

    How to fix this ?

    ReplyDelete
    Replies
    1. It is working fine at my end. Try with any other group -
      ids <- searchGroup(name="data.analytics", token=fb_oauth)

      Delete
  16. This comment has been removed by the author.

    ReplyDelete
  17. This all happen when you turn on this feature, go to your account setting-->account security--> mark check on log in approvals.
    Click here

    ReplyDelete
  18. Very Nice Information Thanks For Sharing It With All Of Us.

    Please Visit Getintopc

    ReplyDelete
  19. I am getting below error after running the authentication script, on browser:

    "Can't load URL: The domain of this URL isn't included in the app's domains. To be able to load this URL, add all domains and sub-domains of your app to the App Domains field in your app settings"

    ReplyDelete
  20. Just a quick question, may I do status update for a particular FB page? If yes, let me know how is it possible. thanks.

    ReplyDelete
  21. Thanks for this article very helpful though not working anymore. To what I understand looking into other blogs

    Facebook has imposed the use of the https:// instead http://. So http://localhost:1410 can't work and gives error.

    Any help would be greatly appreciated.


    ReplyDelete
  22. Waiting for authentication in browser...
    Press Esc/Ctrl + C to abort

    after this my browser is not opening... any solution....

    ReplyDelete
  23. When i CTRL+ENTER in R Studio. It goes to broeser and shows below error on browser:

    Insecure Login Blocked: You can't get an access token or log in to this app from an insecure page. Try re-loading the page as https://

    And in R studio cansole shows below error:

    Authentication complete.
    Error in oauth2.0_access_token(endpoint, app, code = code, user_params = user_params, :
    Bad Request (HTTP 400). Failed to get an access token.


    Please help me to solve this.

    ReplyDelete
  24. Thank you because you have been willing to share information with us. we will always appreciate all you have done here because I know you are very concerned with our. Buy Facebook Likes

    ReplyDelete
  25. This comment has been removed by the author.

    ReplyDelete
  26. Error in fbOAuth(app_id = "2***************5", app_secret = "8****************d", :
    Rfacebook requires httr version 1.2.0 or greater

    ReplyDelete
  27. Greetings, have you ever before wondered to write regarding Nintendo Dsi handheld? Increase facebook video views

    ReplyDelete
  28. So you want to increase your facebook live viewers Buy Facebook Live Viewers , you say. You can buy your facebook live viewers from us and your feed will be popular! Get more attention to your live feeds!

    ReplyDelete
  29. Yes, it is very important as this is the period when you should need to have the best and relaxing life to deliver a healthy baby.  Buy Facebook Love Reactions

    ReplyDelete
  30. Nice Post.Thanks For Sharing This good information.filehippy

    ReplyDelete
  31. hi i can't Extract a list of users who liked the maximum liked posts

    ReplyDelete
  32. First I would like to thank you for this wonderful and helpful tutorial. Now I am facing a problem; I followed your steps and I can extract information about me. However, when I run the following line:

    searchPages( string="trump", token=fb_oauth, n=200)

    This error message occurs:

    Error in callAPI(url = url, token = token) :
    (#10) To use 'Page Public Content Access', your use of this endpoint must be reviewed and approved by Facebook. To submit this 'Page Public Content Access' feature for review please read our documentation on reviewable features: https://developers.facebook.com/docs/apps/review.

    What did I wrong? I am newbie in web - scrapping so would you please enlighten me?

    Thanks in advance.

    ReplyDelete
  33. i get this error:
    Copy and paste into Site URL on Facebook App Settings: http://localhost:1410/
    When done, press any key to continue...
    Waiting for authentication in browser...
    Press Esc/Ctrl + C to abort
    Authentication complete.
    Error in curl::curl_fetch_memory(url, handle = handle) :
    Timeout was reached: Operation timed out after 10016 milliseconds with 0 out of 0 bytes received

    ReplyDelete
    Replies
    1. it means authentification complete but it was'nt succesful
      how can i fix this error? it is OAUth v.3

      Delete
  34. Is it possible to know the post to which a particular Facebook id reacts??

    ReplyDelete
  35. it's a very useful tutorial! it helped me a lot, thank you very much

    ReplyDelete
  36. Instagram is quite popular with everybody these days. It gives all those who are on it an opportunity to display their prized photographs to the world. A feather in its cap, Instagram allows you to add pictures in a timely fashion. What's more is you can make use of hash tags to classify the pictures. Adding the appropriate keywords lets your clients see what they want to see.For you to be popular on Instagram, you are required to have to have many followers. The more the better. One way to do this is to buy followers. This can help you go viral almost instantly.
    http://followersuk.co.uk/

    ReplyDelete
  37. If you want to get famous and promote your business then you need to enhance your Profile visibility on Instagram by gaining your followers, likes, and comments from Hubsoon.com.
    Buy Instagram Followers UK

    ReplyDelete
  38. Great post thank you so much for update,,,, more info please visit my website Buy real instagram likes

    ReplyDelete
  39. LinkedIn indicates 57,910 outcomes for "online networking supervisor" social networking site

    ReplyDelete
  40. Thanks for sharing. Kanhasoft is CRM Software Development Company in India and USA. CRM software allows to gain a profound insight into the business processes, customer contact management, complain management, marketing campaigns, and sales analytics.

    ReplyDelete
  41. Awesome post, and incredible site. Much obliged for the data! emoticon vs emoji

    ReplyDelete

Next → ← Prev