Python Web Scraping : Extract Data from Website and Save it as CSV

Rakesh singhania
2 min readOct 24, 2023
Photo by Kevin Jarrett on Unsplash

In this tutorial, we will go through into the process of web scraping, illustrating how to extract data from a website and then convert it into a CSV file using Python.

We will start with a fundamental example that showcases the extraction and transformation of data from a website to the CSV format.

It’s worth noting that this approach is applicable to virtually any website.

For the sake of this learning exercise, we will use the Flipkart website as our sample source. Our aim is to extract data for a single product type and save the result in CSV format.

Prerequisites:

  • Python 3.5 or higher
  • Python editor / Jupyter Notebook
  • pip

Packages to install:

  • beautifulsoup4
  • requests
  • pandas

Steps:

  1. Import the required packages:
from bs4 import BeautifulSoup
import requests
import pandas as pd
  • Send a request to the Flipkart website and get the HTML code:
URL = "https://www.flipkart.com/search?q=badminton&as-searchtext=badminton"response = requests.get(url)
html_code = response.content
  • Parse the HTML code using BeautifulSoup:
soup = BeautifulSoup(html_code, "html.parser")
  • Extract the desired data from the HTML code:
# sample to get price

d = {"title": [], "price": [], "originalPrice": [], "off %": [], "rating": [], "reviews": [], "availability": [],
"link": []}

def get_price(soup):
try:
price = soup.find("div", attrs={'class': '_30jeq3 _16Jk6d'}).string.strip()
except AttributeError:
price = 0
return price
  • Create a Pandas DataFrame from the extracted data:

flipkart_df = pd.DataFrame.from_dict(d)
  • Save the DataFrame to a CSV file:

--

--

Rakesh singhania

As a student of technology, each day I take a single step forward on the path of learning.