Handle Cookies in RSelenium

In web scraping there are many times you need to play with cookies to extract data from website. In web world, cookies refers to a small piece of data stored on the user's computer by the web browser while browsing a website. They record user's browsing information and later it can be recalled. It is mainly used for web analytics and digital ad campaigns. For example how many users visited a particular link and how many of them clicked on the ad. Ecommerce companies also use this to track information of customers who added an item in the cart but transaction has not been made yet.

Selenium : Install and Load in R

Make sure selenium is already installed and loaded. In R you can install selenium by using RSelenium package. Also you need to install chrome webdriver for the version of your chrome browser installed on your machine. Once done you can run the commands below to load package and activate the driver.

library(RSelenium)
driver<- rsDriver(browser=c("chrome"), chromever = "86.0.4240.22")
remDr <- driver[["client"]]

Save Cookies

By using remDr$getAllCookies() you can extract all the cookies and later you can save it in RDS format.

remDr$navigate("https://www.zillow.com/homes/")
cookies <- remDr$getAllCookies()
saveRDS(cookies, "cookies.rds")

Delete Cookies

By using remDr$deleteAllCookies() you can delete all the cookies of your browser.

# Delete Cookies
remDr$deleteAllCookies()

Add Cookies

You can add previously saved cookies by using function addCookie().

To add a partcular cookies you can use the syntax below.

remDr$addCookie(name = cookies[[1]][["name"]], value = cookies[[1]][["value"]])

To add all cookies in loop (one by one). cookies.rds was saved previously, we are reading that RDS file.

remDr$navigate("https://www.zillow.com/homes/")
cookies <- readRDS("cookies.rds")
for (i in 1:length(cookies)) {
  remDr$addCookie(name = cookies[[i]][["name"]], value = cookies[[i]][["value"]])
}

Spread the Word!
Share Share Tweet

About Author:
Deepanshu Bhalla

Deepanshu founded ListenData with a simple objective - Make analytics easy to understand and follow. He has over 10 years of experience in data science. During his tenure, he worked with global clients in various domains like Banking, Insurance, Private Equity, Telecom and HR.

While I love having friends who agree, I only learn from those who don't
Let's Get Connected Email LinkedIn

Handle Cookies in RSelenium

Post a Comment