How to Scrape Flipkart Products

Crawlbase - Jan 15 - - Dev Community

This blog was originally posted to Crawlbase Blog
Ever found yourself browsing Flipkart and wondered how you could get insights from all those product listings? With over 200 million registered users and a vast array of electronics and fashion products, Flipkart is one of India's leading e-commerce giants. Or you've imagined tracking trends or comparing prices across various items. According to recent statistics, Flipkart hosts over 150 million products across multiple categories, making it a treasure trove of data to explore. Well, you're in for a treat! Web scraping offers a way to gather and analyze this information, and today, we will scrape Flipkart.

Scraping Flipkart might seem tricky, but with the power of Python, a friendly and popular coding language, and the help of the Crawlbase Crawling API, it becomes a breeze. From understanding the structure of Flipkart's pages to scraping Flipkart pages, this guide will walk you through each step. By the end, armed with data from Flipkart's extensive platform, you'll know how to scrape Flipkart and store and analyze the data you gather.

If you want to head right into the steps, click here.

Table Of Contents

  1. Why Scrape Flipkart?
  2. Understanding Flipkart's Structure
  • Layout and Structure of Flipkart Search Pages
  • Key Data Points to Extract
  1. Setting up Your Environment
  • Installing Python and required libraries
  • Choosing the Right Development IDE
  • Crawlbase Registration and API Token
  1. Scraping Flipkart Products
  • Crafting the URL for Targeted Scraping
  • Fetching HTML of Web Page
  • Inspecting HTML to Get CSS Selectors
  • Extracting Product Details
  • Handling Pagination for Multiple Product Pages
  1. Storing the Scraped Data
  • Storing Scraped Data in CSV File
  • Store Scraped Data in SQLite Database
  1. Case Study: Analyzing Flipkart Trends
  • Visualizing Data using Python Libraries (Matplotlib, Seaborn)
  • Drawing Insights from Scraped Data
  1. Final Words
  2. Frequently Asked Questions

Why Scrape Flipkart?

Here’s why you should scrape Flipkart:

Image description

A. Uncover Hidden Insights:

Thousands of products are listed, sold, and reviewed daily on Flipkart. By scraping this data, you can uncover trends, understand customer preferences, and identify emerging market demands. For businesses, this means making informed decisions about product launches, pricing strategies, and marketing campaigns.

B. Competitive Analysis:

Scraping allows you to keep a pulse on the competition. You can position your offerings more competitively by monitoring prices, product availability, and customer reviews. Furthermore, understanding your competitors' strengths and weaknesses can pave the way for strategic advantages in the market.

C. Personalized Shopping Experiences:

For consumers, scraping can lead to a more personalized shopping journey. By analyzing product reviews, ratings, and descriptions, e-commerce platforms can offer tailored product recommendations, ensuring that customers find what they're looking for faster and with greater satisfaction.

D. Enhanced Product Research:

For sellers and manufacturers, scraping Flipkart provides a wealth of product data. From understanding which features customers value the most to gauging market demand for specific categories, this data can be instrumental in guiding product development and innovation.

E. Stay Updated with Market Dynamics:

The e-commerce landscape is dynamic, with products being regularly added, sold out, or discounted. Scraping Flipkart ensures you're constantly updated with these changes, allowing for timely responses and proactive strategies.

Understanding Flipkart's Structure

For web scraping Flipkart you need to understand the layout and organization of the website to extract valuable information. Let's look into Flipkart's structure.

Layout and Structure of Flipkart Search Pages

Flipkart's search pages are meticulously designed to offer users a seamless shopping experience. However, beneath the user-friendly interface lies a structured arrangement of HTML elements that house the product listings, reviews, prices, and more.

Image description

  1. Header: Dominating the top section, you'll encounter the iconic Flipkart logo, the omnipresent search bar, and intuitive navigation links leading to diverse categories and specialized sections.
  2. Search Results: The heart of the page showcases product listings in a structured format. A captivating image, a brief title, and crucial details such as price and ratings accompany each product.
  3. Pagination: Flipkart incorporates a pagination mechanism at the page's bottom for expansive searches, facilitating easy navigation across multiple search result pages.
  4. Reviews for Popular Items: Scrolling further down reveals a dedicated section highlighting popular items in your search criteria. This segment provides insights into trending products and their reception among buyers.
  5. Footer: As the page's foundation, the footer houses essential links, policies, and additional resources, ensuring users can access all pertinent information without cluttering the main interface.

Key Data Points to Extract from Flipkart

When scraping Flipkart, specific data points stand out as crucial:

  1. Product Name: The name or title of the product.
  2. Price: The current price of the product listing.
  3. Rating: The average rating given by users for the product.
  4. Review Count: The number of reviews or rating users give for the product.
  5. Product Category: The category or section to which the product belongs.

By understanding these elements and their arrangement on the search pages, you'll be better equipped to extract the data you need from Flipkart.

Setting up Your Environment

Before scraping Flipkart product listings, we must ensure our setup is ready. We must install the tools and libraries needed, pick the right IDE, and get the critical API credentials.

Installing Python and Required Libraries

  • The first step in setting up your environment is to ensure you have Python installed on your system. If you still need to install Python, download it from the official website at python.org.

  • Once you have Python installed, the next step is to make sure you have the required libraries for this project. In our case, we'll need three main libraries:

    • Crawlbase Python Library: This library will be used to make HTTP requests to the FlipKart search page using the Crawlbase Crawling API. To install it, you can use pip with the following command: 
  pip install crawlbase
Enter fullscreen mode Exit fullscreen mode
  • Beautiful Soup 4: Beautiful Soup is a Python library that makes it easy to scrape and parse HTML content from web pages. It's a critical tool for extracting data from the web. You can install it using pip: 
  pip install beautifulsoup4
Enter fullscreen mode Exit fullscreen mode
  • Pandas: Pandas is a powerful data manipulation and analysis library in Python. We'll use it to store and manage the scraped data. Install pandas with pip: 
  pip install pandas
Enter fullscreen mode Exit fullscreen mode
  • Matplotlib: A fundamental plotting library in Python, it's essential for visualizing data and creating various types of plots and charts. You can install it using pip: 
  pip install matplotlib
Enter fullscreen mode Exit fullscreen mode
  • Seaborn: Built on top of Matplotlib, Seaborn provides a high-level interface for creating attractive and informative statistical graphics. Enhance your visualizations further by installing it via pip: 
  pip install seaborn
Enter fullscreen mode Exit fullscreen mode

Choosing the Right Development IDE

An Integrated Development Environment (IDE) provides a coding environment with features like code highlighting, auto-completion, and debugging tools. While you can write Python code in a simple text editor, an IDE can significantly improve your development experience.

Here are a few popular Python IDEs to consider:

  1. PyCharm: PyCharm is a robust IDE with a free Community Edition. It offers features like code analysis, a visual debugger, and support for web development.

  2. Visual Studio Code (VS Code): VS Code is a free, open-source code editor developed by Microsoft. Its vast extension library makes it versatile for various programming tasks, including web scraping.

  3. Jupyter Notebook: Jupyter Notebook is excellent for interactive coding and data exploration. It's commonly used in data science projects.

  4. Spyder: Spyder is an IDE designed for scientific and data-related tasks. It provides features like a variable explorer and an interactive console.

Crawlbase Registration and API Token

To use the Crawlbase Crawling API for making HTTP requests to the Flipkart search page, you must sign up for an account on the Crawlbase website. Now, let's get you set up with a Crawlbase account. Follow these steps:

  1. Visit the Crawlbase Website: Open your web browser and navigate to the Crawlbase website Signup page to begin the registration process.
  2. Provide Your Details: You'll be asked to provide your email address and create a password for your Crawlbase account. Fill in the required information.
  3. Verification: After submitting your details, you may need to verify your email address. Check your inbox for a verification email from Crawlbase and follow the instructions provided.
  4. Login: Once your account is verified, return to the Crawlbase website and log in using your newly created credentials.
  5. Access Your API Token: You'll need an API token to use the Crawlbase Crawling API. You can find your tokens here.

Note: Crawlbase offers two types of tokens, one for static websites and another for dynamic or JavaScript-driven websites. Since we're scraping Flipkart, we'll opt for the Normal Token. Crawlbase generously offers an initial allowance of 1,000 free requests for the Crawling API, making it an excellent choice for our web scraping project.

With Python and the required libraries installed, the IDE of your choice set up, and your Crawlbase token in hand, you're well-prepared to start scraping Flipkart products.

Scraping Flipkart Products

Let's get into the details of how you can scrape Flipkart. Each step will be simplified with Python code examples provided to enhance clarity.

Crafting the URL for Targeted Scraping

Think of the URL as the webpage address you want to visit. To get the right products, you need the right address. Here's a simple example of how you might create a Flipkart URL for searching mobile phones:

search_query = "headphone"
url = f"https://www.flipkart.com/search?q={search_query}"
Enter fullscreen mode Exit fullscreen mode

search_query = "headphone"
url = f"https://www.flipkart.com/search?q={search_query}"

Fetching HTML of Web Page

To extract data from a webpage, the first step is to fetch its HTML content. Using Python, particularly with the crawlbase library's Crawling API, this process becomes straightforward. Here's a simple example that demonstrates how to use the crawlbase library to fetch the HTML content of a webpage:

from crawlbase import CrawlingAPI

# Initialize the CrawlingAPI with your API token
API_TOKEN = 'YOUR_CRAWLBASE_TOKEN'
crawlbase_api = CrawlingAPI({'token': API_TOKEN})

search_query = "headphone"
url = f"https://www.flipkart.com/search?q={search_query}"

# Use the CrawlingAPI to fetch the webpage content
response = crawlbase_api.get(url)

# Check if the request was successful (status code 200)
if response['status_code'] == 200:
    # Decode the HTML content from bytes to string
    html_content = response['body'].decode('latin1')
    print(html_content)
else:
    print(f"Failed to fetch the webpage. Status code: {response['status_code']}")
Enter fullscreen mode Exit fullscreen mode

To initiate the Flipkart scraping process, follow these straightforward steps:

  1. Create the Script: Begin by creating a new Python script file. Name it flipkart_scraping.py.
  2. Paste the Code: Copy the above code and paste it into your newly created flipkart_scraping.py file. Make sure you add your token.
  3. Execution: Open your command prompt or terminal.
  4. Run the Script: Navigate to the directory containing flipkart_scraping.py and execute the script using the following command: 
python flipkart_scraping.py
Enter fullscreen mode Exit fullscreen mode

Upon execution, the HTML content of the page will be displayed in your terminal.

Image description

Inspecting HTML to Get CSS Selectors

With the HTML content obtained from the search page, the next step is to analyze its structure and pinpoint the location of pricing data. This task is where web development tools and browser developer tools come to our rescue. Let's outline how you can inspect the HTML structure and unearth those precious CSS selectors:

Image description

  1. Open the Web Page: Navigate to the Flipkart website and land on a property page that beckons your interest.
  2. Right-Click and Inspect: Employ your right-clicking prowess on an element you wish to extract and select "Inspect" or "Inspect Element" from the context menu. This mystical incantation will conjure the browser's developer tools.
  3. Locate the HTML Source: Within the confines of the developer tools, the HTML source code of the web page will lay bare its secrets. Hover your cursor over various elements in the HTML panel and witness the corresponding portions of the web page magically illuminate.
  4. Identify CSS Selectors: To liberate data from a particular element, right-click on it within the developer tools and gracefully choose "Copy" > "Copy selector." This elegant maneuver will transport the CSS selector for that element to your clipboard, ready to be wielded in your web scraping incantations.

Once you have these selectors, you can proceed to structure your Flipkart scraper to extract the required information effectively.

Extracting Product Details

Once the initial HTML data is obtained, the next task is to derive valuable insights from it. This is where the utility of BeautifulSoup shines through. As a Python module, BeautifulSoup is adept at analyzing HTML and XML files, equipping users with methods to traverse the document structure and locate specific content.

With the assistance of BeautifulSoup, users can identify particular HTML components and retrieve pertinent details. In the following script, BeautifulSoup is employed to extract essential details like the product title, rating, review count, price, and URL link (Product Page URL) for every product listed on the specified Flipkart search page.

from crawlbase import CrawlingAPI
from bs4 import BeautifulSoup
import json
import re

# Initialize the CrawlingAPI with your API token
API_TOKEN = 'YOUR_CRAWLBASE_TOKEN'
crawling_api = CrawlingAPI({'token': API_TOKEN})

def fetch_page_html(url):
    response = crawling_api.get(url)
    if response['status_code'] == 200:
        return response['body'].decode('latin1')
    else:
        print(f"Request failed with status code {response['status_code']}: {response['body']}")
        return None

def extract_products_info(html_content):
    # Parse the HTML content using BeautifulSoup
    soup = BeautifulSoup(html_content, 'html.parser')

    scraped_products_data = []

    # Identify the section or elements containing product listings
    product_listings = soup.select('div#container > div:first-child > div:nth-child(3) > div:first-child > div:nth-child(2) div[data-id]')  # Adjust class or tag as per the webpage structure

    # Iterate through each product listing and extract desired information
    for product in product_listings:
        title_element = product.select_one('a[title]')
        rating_element = product.select_one('span[id^="productRating"] > div')
        review_count_element = product.select_one('span[id^="productRating"] + span')
        price_element = product.select_one('a._8VNy32 > div > div:first-child')
        url_link_element = product.select_one('a[title]')

        product_data = {
            "Title": title_element['title'].strip() if title_element else None,
            "Rating": rating_element.text.strip() if rating_element else None,
            "Review Count": review_count_element.text.strip() if review_count_element else None,
            "Price": re.sub(r'\D', '', price_element.text.strip()) if price_element else None,
            "URL Link": "https://www.flipkart.com" + url_link_element['href'].strip() if url_link_element else None,
        }

        scraped_products_data.append(product_data)

    return scraped_products_data


if __name__ == "__main__":
    search_query = "headphone"
    url = f"https://www.flipkart.com/search?q={search_query}"
    html_content = fetch_page_html(url)

    if html_content:
        products_data = extract_products_info(html_content)
        print(json.dumps(products_data, indent=2))
Enter fullscreen mode Exit fullscreen mode

In the example above:

  • We utilize the crawlbase library for fetching the webpage HTML content.
  • After fetching the webpage content, we parse it using BeautifulSoup library.
  • We then identify the specific sections or elements that contain product listings.
  • For each product listing, we extract relevant details such as the title, rating, review count, price, and URL link.

Output:

[
  {
    "Title": "truke Buds S1 with Quad Mic ENC, 72 Hours Playtime, 10mm drivers with AAC codec Bluetooth Headset",
    "Rating": "4",
    "Review Count": "(65,375)",
    "Price": "1399",
    "URL Link": "https://www.flipkart.com/truke-buds-s1-quad-mic-enc-72-hours-playtime-10mm-drivers-aac-codec-bluetooth-headset/p/itma77ffe347c036?pid=ACCGFGCPUJMRBGK6&lid=LSTACCGFGCPUJMRBGK64XNR12&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_1&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN6SmNQxzJpFvFyiNOHpu92ALdUpPiMmgBd-Hqs9vGF1Tg%3D%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Microflash S7-Deep Bass, Clear Hi-Fi Sound, Headphones Wired Headset",
    "Rating": "3.8",
    "Review Count": "(11,982)",
    "Price": "249",
    "URL Link": "https://www.flipkart.com/microflash-s7-deep-bass-clear-hi-fi-sound-headphones-wired-headset/p/itm822e3564415fd?pid=ACCGHEYDFRXHWMGC&lid=LSTACCGHEYDFRXHWMGCT7NUM8&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_2&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN6mF3QSCiKNVC3UKI8D6RebGyyJfiAiWSqn_PPpOr8H7g%3D%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "realme Buds 2 Wired Headset",
    "Rating": "4.4",
    "Review Count": "(7,37,218)",
    "Price": "599",
    "URL Link": "https://www.flipkart.com/realme-buds-2-wired-headset/p/itm393326c26b6ad?pid=ACCFKYE2ARGG67WC&lid=LSTACCFKYE2ARGG67WCEU0OSD&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&spotlightTagId=FkPickId_0pm%2Ffcn&srno=s_1_3&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCFKYE2ARGG67WC.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Mivi DuoPods K2 TWS,AI-ENC,40Hr Playtime,13mm Bass,Made in India Bluetooth Headset",
    "Rating": "3.9",
    "Review Count": "(33,386)",
    "Price": "799",
    "URL Link": "https://www.flipkart.com/mivi-duopods-k2-tws-ai-enc-40hr-playtime-13mm-bass-made-india-bluetooth-headset/p/itm51fbf520efdbd?pid=ACCGPFCNADSRMQGJ&lid=LSTACCGPFCNADSRMQGJ2GCXC1&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_4&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGPFCNADSRMQGJ.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "OnePlus Bullets Wireless Z2 with Fast Charge, 30 Hrs Battery Life, Earphones with mic Bluetooth Headset",
    "Rating": "4.3",
    "Review Count": "(9,05,873)",
    "Price": "1799",
    "URL Link": "https://www.flipkart.com/oneplus-bullets-wireless-z2-fast-charge-30-hrs-battery-life-earphones-mic-bluetooth-headset/p/itm1b9cd98911a2a?pid=ACCGC4GN5SQR7ZCZ&lid=LSTACCGC4GN5SQR7ZCZYKXWJM&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&spotlightTagId=BestsellerId_0pm%2Ffcn&srno=s_1_5&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGC4GN5SQR7ZCZ.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Noise Buds VS102 with 50 Hrs Playtime, 11mm Driver, IPX5 and Unique Flybird Design Bluetooth Headset",
    "Rating": "4",
    "Review Count": "(4,77,818)",
    "Price": "799",
    "URL Link": "https://www.flipkart.com/noise-buds-vs102-50-hrs-playtime-11mm-driver-ipx5-unique-flybird-design-bluetooth-headset/p/itm95562396a49a7?pid=ACCG4RBWG8TWZRY9&lid=LSTACCG4RBWG8TWZRY9VZWQNM&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_6&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCG4RBWG8TWZRY9.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "OnePlus Nord Buds 2 True Wireless Earbuds with 25dB Active Noise Cancellation Bluetooth Headset",
    "Rating": "4.3",
    "Review Count": "(43,603)",
    "Price": "2699",
    "URL Link": "https://www.flipkart.com/oneplus-nord-buds-2-true-wireless-earbuds-25db-active-noise-cancellation-bluetooth-headset/p/itm1564d0b502ff6?pid=ACCGNSJ2TZEJ3U66&lid=LSTACCGNSJ2TZEJ3U66DYYFN0&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_7&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGNSJ2TZEJ3U66.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Microflash S7-Deep Bass, Clear Hi-Fi Sound, Headphones Wired Headset",
    "Rating": "3.8",
    "Review Count": "(11,982)",
    "Price": "249",
    "URL Link": "https://www.flipkart.com/microflash-s7-deep-bass-clear-hi-fi-sound-headphones-wired-headset/p/itme66ddf4e418b8?pid=ACCGHE4EWZZJNF2Q&lid=LSTACCGHE4EWZZJNF2QJ8HPIR&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_8&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN69Uuz8KqWNN_i11viNZRWr5bqTZslnFLbm1Haf7EmtBg%3D%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "APPLE AirPods Pro (2nd generation) with MagSafe Case (USB-C) Bluetooth Headset",
    "Rating": "4.6",
    "Review Count": "(1,700)",
    "Price": "20990",
    "URL Link": "https://www.flipkart.com/apple-airpods-pro-2nd-generation-magsafe-case-usb-c-bluetooth-headset/p/itm60c8f5a308352?pid=ACCGTCYRNADXCXNG&lid=LSTACCGTCYRNADXCXNGWCU03Q&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_9&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGTCYRNADXCXNG.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "OnePlus Bullets Wireless Z2 ANC Bluetooth in Ear Earphones with 45dB Hybrid ANC Bluetooth Headset",
    "Rating": "4.2",
    "Review Count": "(22,921)",
    "Price": "1999",
    "URL Link": "https://www.flipkart.com/oneplus-bullets-wireless-z2-anc-bluetooth-ear-earphones-45db-hybrid-headset/p/itmf053b9e0ed84a?pid=ACCGRH9DVGQCF5CF&lid=LSTACCGRH9DVGQCF5CFOI349Z&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_10&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGRH9DVGQCF5CF.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "truke Buds Q1 with 30H Playtime, Quad Mic ENC, Game Mode, 10mm driver, AAC codec Bluetooth Headset",
    "Rating": "4",
    "Review Count": "(75,999)",
    "Price": "1199",
    "URL Link": "https://www.flipkart.com/truke-buds-q1-30h-playtime-quad-mic-enc-game-mode-10mm-driver-aac-codec-bluetooth-headset/p/itm86e5f0262925f?pid=ACCGFGCPJBYDY9KX&lid=LSTACCGFGCPJBYDY9KXXIVOVX&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_11&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN7JDdHlWlPDWb1KqP-u5jXeDjgzLp_2btDaESZxNFnPAUhz9c-MSIoFe8xMxkfU6dM%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Microflash S7-Deep Bass, Clear Hi-Fi Sound, Headphones Wired Headset",
    "Rating": "3.8",
    "Review Count": "(11,982)",
    "Price": "249",
    "URL Link": "https://www.flipkart.com/microflash-s7-deep-bass-clear-hi-fi-sound-headphones-wired-headset/p/itm15ad7438eacba?pid=ACCGHE5F8Q4G3YNZ&lid=LSTACCGHE5F8Q4G3YNZN9NXMM&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_12&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN60L66IpvDw7j6EmbWRkT0D2weRQgAOe-a1uDhmYkAX5Q%3D%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "realme Techlife Buds T100 with up to 28 Hours Playback & AI ENC for Calls Bluetooth Headset",
    "Rating": "4.2",
    "Review Count": "(1,72,222)",
    "Price": "1399",
    "URL Link": "https://www.flipkart.com/realme-techlife-buds-t100-up-28-hours-playback-ai-enc-calls-bluetooth-headset/p/itmb496242b3d1fb?pid=ACCGGZEH4K3ZCQFT&lid=LSTACCGGZEH4K3ZCQFTNO8HGO&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_13&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGGZEH4K3ZCQFT.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "OnePlus Bullets Wireless Z2 Bluetooth Headset",
    "Rating": "4.3",
    "Review Count": "(9,05,873)",
    "Price": "1799",
    "URL Link": "https://www.flipkart.com/oneplus-bullets-wireless-z2-bluetooth-headset/p/itm4c3852314bb61?pid=ACCGF4JZHYTERVTY&lid=LSTACCGF4JZHYTERVTYHCZ96N&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_14&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGF4JZHYTERVTY.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "realme Techlife Buds T100 with up to 28 Hours Playback & AI ENC for Calls Bluetooth Headset",
    "Rating": "4.2",
    "Review Count": "(1,72,222)",
    "Price": "1399",
    "URL Link": "https://www.flipkart.com/realme-techlife-buds-t100-up-28-hours-playback-ai-enc-calls-bluetooth-headset/p/itm5cf0046c3e206?pid=ACCGH32HGHQHPFES&lid=LSTACCGH32HGHQHPFESW65AP4&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_15&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGH32HGHQHPFES.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "OPPO Enco Buds 2 with 28 hours Battery life & Deep Noise Cancellation Bluetooth Headset",
    "Rating": "4.2",
    "Review Count": "(2,66,046)",
    "Price": "1999",
    "URL Link": "https://www.flipkart.com/oppo-enco-buds-2-28-hours-battery-life-deep-noise-cancellation-bluetooth-headset/p/itm3344fa26518ed?pid=ACCGH7YZY6AHGCHJ&lid=LSTACCGH7YZY6AHGCHJHLWF41&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_16&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGH7YZY6AHGCHJ.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "truke Clarity 6 with 80Hr Battery, 6Mic Adv.ENC, Dual Connectivity, Touch Glass, 5.3v Bluetooth Headset",
    "Rating": "4",
    "Review Count": "(45)",
    "Price": "1299",
    "URL Link": "https://www.flipkart.com/truke-clarity-6-80hr-battery-6mic-adv-enc-dual-connectivity-touch-glass-5-3v-bluetooth-headset/p/itmb941e955ea6a0?pid=ACCGT3JKJ8TRWYXM&lid=LSTACCGT3JKJ8TRWYXMYY5N29&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_17&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN7aX0MIN49HChZYth_QiLPBf0TWsMeuzw0YcVtNCbDfJ2Vkv0XZFiOX7bbXdlao_f8%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Cellecor BroPods CB05 Waterproof Earbuds With 25 Hours Playtime, Auto Pairing, 13mm Diver Bluetooth Headset",
    "Rating": "3.8",
    "Review Count": "(32)",
    "Price": "799",
    "URL Link": "https://www.flipkart.com/cellecor-bropods-cb05-waterproof-earbuds-25-hours-playtime-auto-pairing-13mm-diver-bluetooth-headset/p/itm1df80effd7950?pid=ACCGKHZ6N7GVQXPA&lid=LSTACCGKHZ6N7GVQXPAEBLNIM&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_18&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN7cqyxXl9oqlXZ_vPdXo44vg3HYu5uc1-tkmCattHIeUg%3D%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Noise Buds VS102 with 50 Hrs Playtime, 11mm Driver, IPX5 and Unique Flybird Design Bluetooth Headset",
    "Rating": "4",
    "Review Count": "(4,77,818)",
    "Price": "799",
    "URL Link": "https://www.flipkart.com/noise-buds-vs102-50-hrs-playtime-11mm-driver-ipx5-unique-flybird-design-bluetooth-headset/p/itm6cc239faac681?pid=ACCGBWZ7SZUHJGC8&lid=LSTACCGBWZ7SZUHJGC8FS7HET&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_19&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGBWZ7SZUHJGC8.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Boult Audio W20 with Zen ENC Mic, 32H Battery Life, Low Latency Gaming, Made in India, 5.3v Bluetooth Headset",
    "Rating": "4.1",
    "Review Count": "(20,984)",
    "Price": "899",
    "URL Link": "https://www.flipkart.com/boult-audio-w20-zen-enc-mic-32h-battery-life-low-latency-gaming-made-india-5-3v-bluetooth-headset/p/itm4255cf81a0d8d?pid=ACCGRNFM5GBPUKPF&lid=LSTACCGRNFM5GBPUKPFYSZIXK&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_20&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGRNFM5GBPUKPF.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "TEMPT Thunder Wireless Bluetooth Earbuds with Massive 24 Hrs Total Playtime Bluetooth Gaming Headset",
    "Rating": "4.4",
    "Review Count": "(20)",
    "Price": "1399",
    "URL Link": "https://www.flipkart.com/tempt-thunder-wireless-bluetooth-earbuds-massive-24-hrs-total-playtime-gaming-headset/p/itm7ffda64fbcc8b?pid=ACCGHZ5FFXHP472H&lid=LSTACCGHZ5FFXHP472HKCCYI2&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_21&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN46AL8slpotsF2mGRZNXvuco5cW0zhD8rK5C7_Cp3YDbA%3D%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "TEMPT Thunder Wireless Bluetooth Earbuds with Massive 24 Hrs Total Playtime Bluetooth Gaming Headset",
    "Rating": "4.4",
    "Review Count": "(20)",
    "Price": "1399",
    "URL Link": "https://www.flipkart.com/tempt-thunder-wireless-bluetooth-earbuds-massive-24-hrs-total-playtime-gaming-headset/p/itm022f31f1912dc?pid=ACCGHZ5534JQNU9G&lid=LSTACCGHZ5534JQNU9GPHTIGR&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_22&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN5sZd52gh8qWZpNoMi6O7vK64nsyOxHu4f0l6QTN68Ryg%3D%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "realme Buds Wireless 3 with 30dB ANC, 360 degree Spatial Audio, upto 40 hours Playback Bluetooth Headset",
    "Rating": "4.3",
    "Review Count": "(41,337)",
    "Price": "1699",
    "URL Link": "https://www.flipkart.com/realme-buds-wireless-3-30db-anc-360-degree-spatial-audio-upto-40-hours-playback-bluetooth-headset/p/itmecb3974dcf5aa?pid=ACCGQXVJNVBJH9D8&lid=LSTACCGQXVJNVBJH9D8VM3IEJ&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_23&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGQXVJNVBJH9D8.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "realme Buds Wireless 2 Neo with Type-C Fast Charge & Bass Boost+ Bluetooth Headset",
    "Rating": "4.2",
    "Review Count": "(3,64,554)",
    "Price": "1299",
    "URL Link": "https://www.flipkart.com/realme-buds-wireless-2-neo-type-c-fast-charge-bass-boost-bluetooth-headset/p/itm342ebcfc6e12b?pid=ACCG65YC8DTGDCNN&lid=LSTACCG65YC8DTGDCNNZ8HHUD&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_24&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCG65YC8DTGDCNN.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "realme Buds T300 with 12.4mm Driver, 30dB ANC, 360 Spatial Audio and 40 hours Playback Bluetooth Headset",
    "Rating": "4.3",
    "Review Count": "(17,538)",
    "Price": "2299",
    "URL Link": "https://www.flipkart.com/realme-buds-t300-12-4mm-driver-30db-anc-360-spatial-audio-40-hours-playback-bluetooth-headset/p/itm340c420d1734a?pid=ACCGSUP9KHU5VAZJ&lid=LSTACCGSUP9KHU5VAZJTGZSEW&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_25&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGSUP9KHU5VAZJ.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Mivi DuoPods K2 TWS,AI-ENC,40Hr Playtime,13mm Bass,Made in India Bluetooth Headset",
    "Rating": "3.9",
    "Review Count": "(33,386)",
    "Price": "799",
    "URL Link": "https://www.flipkart.com/mivi-duopods-k2-tws-ai-enc-40hr-playtime-13mm-bass-made-india-bluetooth-headset/p/itm23db354491142?pid=ACCGPFCN6ZKBF5MD&lid=LSTACCGPFCN6ZKBF5MDBVLJD5&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_26&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGPFCN6ZKBF5MD.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Zusix AirTone Mini with 30 Hours Music Time, Deep Bass, Wireless TWS Earbuds Bluetooth Headset",
    "Rating": null,
    "Review Count": null,
    "Price": "699",
    "URL Link": "https://www.flipkart.com/zusix-airtone-mini-30-hours-music-time-deep-bass-wireless-tws-earbuds-bluetooth-headset/p/itmb98297ed48e6f?pid=ACCGUXTHZWF2HYAR&lid=LSTACCGUXTHZWF2HYARAKWVKN&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_27&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN6y2L35YpRn031ak2LLXezIabg-EOnojOY1Fu_9R1lELA%3D%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "truke BTG NEO Gaming Earbuds with 6Mic ENC, Dual Connectivity, 80H Big Battery, 5.3v Bluetooth Headset",
    "Rating": "4",
    "Review Count": "(4,647)",
    "Price": "1399",
    "URL Link": "https://www.flipkart.com/truke-btg-neo-gaming-earbuds-6mic-enc-dual-connectivity-80h-big-battery-5-3v-bluetooth-headset/p/itm506480508871c?pid=ACCGQBRTTQ4AHNH2&lid=LSTACCGQBRTTQ4AHNH2YJQMWM&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_28&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN4VuImUuWoFwd7_nuNdWpiZ0fNKyQBx0OvhZEeC-4tzpYQEIsITtCzc4bHaOMTqL08%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Boult Audio Z40 Pro with 100H Battery, Quad Mic ENC, Scratch Proof, Rubber Grip Design, 5.3v Bluetooth Headset",
    "Rating": "4.1",
    "Review Count": "(18,176)",
    "Price": "1399",
    "URL Link": "https://www.flipkart.com/boult-audio-z40-pro-100h-battery-quad-mic-enc-scratch-proof-rubber-grip-design-5-3v-bluetooth-headset/p/itm04e0d4d6f7151?pid=ACCGSZCDZSXEHWW7&lid=LSTACCGSZCDZSXEHWW7316PRN&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_29&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGSZCDZSXEHWW7.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "boAt Airdopes 161 with 40 Hours Playback, ASAP Charge & 10mm Drivers Bluetooth Headset",
    "Rating": "4",
    "Review Count": "(7,31,833)",
    "Price": "999",
    "URL Link": "https://www.flipkart.com/boat-airdopes-161-40-hours-playback-asap-charge-10mm-drivers-bluetooth-headset/p/itm8a7493150ae4a?pid=ACCG6DS7WDJHGWSH&lid=LSTACCG6DS7WDJHGWSH8IRY50&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&spotlightTagId=BestsellerId_0pm%2Ffcn&srno=s_1_30&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCG6DS7WDJHGWSH.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "AXL EP-22 In-Ear | Stereo Wired Earphone | High Bass with in-Line mic Wired Headset",
    "Rating": "4.1",
    "Review Count": "(31)",
    "Price": "269",
    "URL Link": "https://www.flipkart.com/axl-ep-22-in-ear-stereo-wired-earphone-high-bass-in-line-mic-headset/p/itm6489b50a8cfe5?pid=ACCGGQKTBDHCTHWG&lid=LSTACCGGQKTBDHCTHWGMPZQCZ&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_31&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN4yg84pjAoFgRHxlNVRTRaESyDOnjP420YILO6R91xK9g%3D%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Zusix BattleBeat 20 with 40 Hours Music Time, Deep Bass Wireless TWS Earbuds Bluetooth Headset",
    "Rating": null,
    "Review Count": null,
    "Price": "899",
    "URL Link": "https://www.flipkart.com/zusix-battlebeat-20-40-hours-music-time-deep-bass-wireless-tws-earbuds-bluetooth-headset/p/itm0ad6740fe2c47?pid=ACCGUXTHRGCFBSWZ&lid=LSTACCGUXTHRGCFBSWZWA9GTO&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_32&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN4-_0PTqDkcmnxyNJawFh1oCpme_VZULm5PhbNu1VjxAg%3D%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "realme Buds Air 5 Pro with 50dB ANC, 360 Spatial Audio and upto 40 hours Playback Bluetooth Headset",
    "Rating": "4.4",
    "Review Count": "(7,560)",
    "Price": "4499",
    "URL Link": "https://www.flipkart.com/realme-buds-air-5-pro-50db-anc-360-spatial-audio-upto-40-hours-playback-bluetooth-headset/p/itm37345859fc3d0?pid=ACCGS7MBV4AZCRPV&lid=LSTACCGS7MBV4AZCRPVSZX3GC&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_33&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGS7MBV4AZCRPV.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Nothing Ear (2) with Dual chamber sound, Hi-res audio, Smart ANC and Dual connection Bluetooth Headset",
    "Rating": "4.3",
    "Review Count": "(8,990)",
    "Price": "7999",
    "URL Link": "https://www.flipkart.com/nothing-ear-2-dual-chamber-sound-hi-res-audio-smart-anc-connection-bluetooth-headset/p/itm6d79168be59b2?pid=ACCGQFSYJMHPNFEN&lid=LSTACCGQFSYJMHPNFENYB5EX1&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_34&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGQFSYJMHPNFEN.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "boAt Airdopes 161 with 40 Hours Playback, ASAP Charge & 10mm Drivers Bluetooth Headset",
    "Rating": "4",
    "Review Count": "(7,31,833)",
    "Price": "999",
    "URL Link": "https://www.flipkart.com/boat-airdopes-161-40-hours-playback-asap-charge-10mm-drivers-bluetooth-headset/p/itm0611c83d26e85?pid=ACCG6DS9ZH2ECDBG&lid=LSTACCG6DS9ZH2ECDBGFGTF30&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_35&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCG6DS9ZH2ECDBG.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "OnePlus Bullets Wireless Z2 ANC Bluetooth in Ear Earphones with 45dB Hybrid ANC Bluetooth Headset",
    "Rating": "4.2",
    "Review Count": "(22,921)",
    "Price": "1999",
    "URL Link": "https://www.flipkart.com/oneplus-bullets-wireless-z2-anc-bluetooth-ear-earphones-45db-hybrid-headset/p/itm728a3b64f8240?pid=ACCGRH9DJSV7GZCX&lid=LSTACCGRH9DJSV7GZCXZIPRPK&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_36&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGRH9DJSV7GZCX.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "AXL EP-22 In-Ear | Stereo Wired Earphone | High Bass with in-Line mic Wired Headset",
    "Rating": "3.9",
    "Review Count": "(32)",
    "Price": "258",
    "URL Link": "https://www.flipkart.com/axl-ep-22-in-ear-stereo-wired-earphone-high-bass-in-line-mic-headset/p/itm524dda0827b29?pid=ACCGGQKHNZ4MZNRG&lid=LSTACCGGQKHNZ4MZNRGSQJEPY&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_37&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN70SoLGaIJmG01ceViqwYm1rJeMqnBFTJrzWjSfFCwnTg%3D%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "Boston Levin White Bluetooth TWS, Truely Wireless Earbuds with Upto 32 Hour Playtime STORM 9 Bluetooth Headset",
    "Rating": "4.5",
    "Review Count": "(43)",
    "Price": "1119",
    "URL Link": "https://www.flipkart.com/boston-levin-white-bluetooth-tws-truely-wireless-earbuds-upto-32-hour-playtime-storm-9-headset/p/itm575e794a9e2e3?pid=ACCGSD3WGYVR7BWG&lid=LSTACCGSD3WGYVR7BWGAK2H0Z&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_38&otracker=search&fm=organic&iid=en_F_c0OqFRvb4K-AWSUd-Eq3H84LKxJ_eUsXxQjZROJN7rwmQUE9hrVFspXC6U0BuRoPo4-jSIPNWms5g7fZATPQ%3D%3D&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "realme Buds Wireless 2 Neo with Type-C Fast Charge & Bass Boost+ Bluetooth Headset",
    "Rating": "4.2",
    "Review Count": "(3,64,554)",
    "Price": "1299",
    "URL Link": "https://www.flipkart.com/realme-buds-wireless-2-neo-type-c-fast-charge-bass-boost-bluetooth-headset/p/itm64e67d64bd846?pid=ACCG65YCGHHHZPNZ&lid=LSTACCG65YCGHHHZPNZT5H66U&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_39&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCG65YCGHHHZPNZ.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  },
  {
    "Title": "boAt Nirvana Ion 32dB Active Noise Cancellation, 120HRS, Bionic Mode, HIFI5 Bluetooth Headset",
    "Rating": "4.2",
    "Review Count": "(26,869)",
    "Price": "2499",
    "URL Link": "https://www.flipkart.com/boat-nirvana-ion-32db-active-noise-cancellation-120hrs-bionic-mode-hifi5-bluetooth-headset/p/itmbdf03ab73bf93?pid=ACCGT3NUWVDFDF3W&lid=LSTACCGT3NUWVDFDF3WOTPFK6&marketplace=FLIPKART&q=headphone&store=0pm%2Ffcn&srno=s_1_40&otracker=search&fm=organic&iid=6cd3e772-34c4-4426-ab20-12958545621c.ACCGT3NUWVDFDF3W.SEARCH&ppt=None&ppn=None&ssid=c7fyiaqbg00000001704637783375&qH=b052e360817fdeec"
  }
]
Enter fullscreen mode Exit fullscreen mode

Handling Pagination for Multiple Product Pages

Flipkart's search results are often divided across multiple pages, each containing a set of product listings. To ensure that we gather a comprehensive dataset, we need to handle pagination. This involves iterating through the results pages and making additional requests as needed. Handling pagination is essential for obtaining a complete view of Flipkart's product listings and ensuring your analysis is based on a comprehensive dataset. Let us update our previous script to handle pagination.

from crawlbase import CrawlingAPI
from bs4 import BeautifulSoup
import re

# Initialize the CrawlingAPI with your API token
API_TOKEN = 'YOUR_CRAWLBASE_TOKEN'
crawling_api = CrawlingAPI({'token': API_TOKEN})

def fetch_page_html(url):
    # ... (unchanged)

def extract_products_info(html_content):
    # ... (unchanged)

if __name__ == "__main__":
    search_query = "headphone"
    base_url = f"https://www.flipkart.com/search?q={search_query}&page="
    page_number = 1
    all_products_data = []

    while True:
        url = base_url + str(page_number)
        html_content = fetch_page_html(url)

        if html_content:
            soup = BeautifulSoup(html_content, 'html.parser')
            # Break the loop if no product listings found
            # limiting to 5 pages for the example
            if not soup.select_one('div#container > div:first-child > div:nth-child(3) > div:first-child > div:nth-child(2) div[data-id]') or page_number > 5:
                print("End of Product listing...")
                break

            products_data = extract_products_info(html_content)
            all_products_data.extend(products_data)

            page_number += 1

    # save/process the data as per your need
Enter fullscreen mode Exit fullscreen mode

Storing the Scraped Data

After successfully scraping data from Flipkart's search pages, the next crucial step is storing this valuable information for future analysis and reference. In this section, we will explore two common methods for data storage: saving scraped data in a CSV file and storing it in an SQLite database. These methods allow you to organize and manage your scraped data efficiently.

Storing Scraped Data in a CSV File

CSV is a widely used format for storing tabular data. It's a simple and human-readable way to store structured data, making it an excellent choice for saving your scraped Flipkart product listing data.

We'll extend our previous web scraping script to include a step for saving the scraped data into a CSV file using the popular Python library, pandas. Here's an updated version of the script:

from crawlbase import CrawlingAPI
from bs4 import BeautifulSoup
import pandas as pd
import re

# Initialize the CrawlingAPI with your API token
API_TOKEN = 'YOUR_CRAWLBASE_TOKEN'
crawling_api = CrawlingAPI({'token': API_TOKEN})

def fetch_page_html(url):
    # ... (unchanged)

def extract_products_info(html_content):
    # ... (unchanged)

def main():
    search_query = "headphone"
    base_url = f"https://www.flipkart.com/search?q={search_query}&page="
    page_number = 1
    all_products_data = []

    while True:
        url = base_url + str(page_number)
        html_content = fetch_page_html(url)

        if html_content:
            soup = BeautifulSoup(html_content, 'html.parser')
            # Break the loop if no product listings found
            # limiting to 5 pages for the example
            if not soup.select_one('div#container > div:first-child > div:nth-child(3) > div:first-child > div:nth-child(2) div[data-id]') or page_number > 5:
                print("End of Product listing...")
                break

            products_data = extract_products_info(html_content)
            all_products_data.extend(products_data)

            page_number += 1

    # Save scraped data as a CSV file
    df = pd.DataFrame(all_products_data)
    df.to_csv('flipkart_product_data.csv', index=False)

if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

In this updated script, we've introduced pandas, a powerful data manipulation and analysis library. After scraping and accumulating the product listing details in the all_product_details list, we create a pandas DataFrame from this data. Then, we use the to_csv method to save the DataFrame to a CSV file named "flipkart_product_data.csv" in the current directory. Setting index=False ensures that we don't save the DataFrame's index as a separate column in the CSV file.

flipkart_product_data.csv Preview:

Image description

Storing Scraped Data in an SQLite Database

If you prefer a more structured and query-friendly approach to data storage, SQLite is a lightweight, serverless database engine that can be a great choice. You can create a database table to store your scraped data, allowing for efficient data retrieval and manipulation. Here's how you can modify the script to store data in an SQLite database:

from crawlbase import CrawlingAPI
from bs4 import BeautifulSoup
import sqlite3
import re

# Initialize the CrawlingAPI with your API token
API_TOKEN = 'YOUR_CRAWLBASE_TOKEN'
crawling_api = CrawlingAPI({'token': API_TOKEN})

def fetch_page_html(url):
    # ... (unchanged)

def extract_products_info(html_content):
    # ... (unchanged)

def create_database():
    conn = sqlite3.connect('flipkart_products.db')
    cursor = conn.cursor()
    cursor.execute('''CREATE TABLE IF NOT EXISTS products (
                      id INTEGER PRIMARY KEY AUTOINCREMENT,
                      title TEXT,
                      rating TEXT,
                      review_count TEXT,
                      price TEXT,
                      url_link TEXT
                    )''')
    conn.commit()
    conn.close()

def save_to_database(data):
    conn = sqlite3.connect('flipkart_products.db')
    cursor = conn.cursor()

    # Create a list of tuples from the data
    data_tuples = [(product['Title'], product['Rating'], product['Review Count'], product['Price'], product['URL Link']) for product in data]

    # Insert data into the products table
    cursor.executemany('''
        INSERT INTO products (title, rating, review_count, price, url_link)
        VALUES (?, ?, ?, ?, ?)
    ''', data_tuples)

    conn.commit()
    conn.close()

def main():
    # Create the database and products table
    create_database()

    search_query = "headphone"
    base_url = f"https://www.flipkart.com/search?q={search_query}&page="
    page_number = 1
    all_products_data = []

    while True:
        url = base_url + str(page_number)
        html_content = fetch_page_html(url)

        if html_content:
            soup = BeautifulSoup(html_content, 'html.parser')
            # Break the loop if no product listings found
            # limiting to 5 pages for the example
            if not soup.select_one('div#container > div:first-child > div:nth-child(3) > div:first-child > div:nth-child(2) div[data-id]') or page_number > 5:
                print("End of Product listing...")
                break

            products_data = extract_products_info(html_content)
            all_products_data.extend(products_data)

            page_number += 1

    # Insert scraped data into the SQLite database
    save_to_database(all_products_data)

if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

In this script, we've added functions for creating the SQLite database and table ( create_database ) and saving the scraped data to the database ( save_to_database ). The create_database function checks if the database and table exist and creates them if they don't. The save_to_database function inserts the scraped data into the 'products' table in an SQLite database named 'flipkart_products.db'.

products Table Preview:

Image description

Case Study: Analyzing Flipkart Trends

Extracting data is merely the first step. The true value emerges when this data is transformed into actionable insights. This section delves into how we can harness the scraped data from Flipkart to visualize trends and draw meaningful conclusions.

Visualizing Data using Python Libraries (Matplotlib, Seaborn)

Python offers an extensive suite of visualization tools, with Matplotlib and Seaborn standing out as prominent choices. These libraries empower analysts to craft compelling visuals that succinctly represent intricate data patterns.

Let's consider an illustrative example where we focus on the 'Price' and 'Rating' attributes from our scraped dataset. By plotting the average price of headphones against their respective ratings, we can discern potential correlations and market preferences. We can update our previous script as below.

from crawlbase import CrawlingAPI
from bs4 import BeautifulSoup
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import re

# Initialize the CrawlingAPI with your API token
API_TOKEN = 'YOUR_CRAWLBASE_TOKEN'
crawling_api = CrawlingAPI({'token': API_TOKEN})

def fetch_page_html(url):
    # ... (unchanged)

def extract_products_info(html_content):
    # ... (unchanged)

def plot_PR_graph(products_data):
    df = pd.DataFrame(products_data)

    # Convert 'Rating' and 'Price' columns to numeric
    df['Rating'] = pd.to_numeric(df['Rating'], errors='coerce')
    df['Price'] = pd.to_numeric(df['Price'], errors='coerce')

    # Drop rows where either 'Rating' or 'Price' is NaN
    df = df.dropna(subset=['Rating', 'Price'])

    # Visualization
    plt.figure(figsize=(10, 6))
    sns.barplot(x="Rating", y="Price", data=df, ci=None)
    plt.title('Average Price of Headphones Across Ratings')
    plt.xlabel('Rating')
    plt.ylabel('Average Price (in ₹)')
    plt.show()

def main():
    search_query = "headphone"
    base_url = f"https://www.flipkart.com/search?q={search_query}&page="
    page_number = 1
    all_products_data = []

    while True:
        url = base_url + str(page_number)
        html_content = fetch_page_html(url)

        if html_content:
            soup = BeautifulSoup(html_content, 'html.parser')
            # Break the loop if no product listings found
            # limiting to 5 pages for the example
            if not soup.select_one('div#container > div:first-child > div:nth-child(3) > div:first-child > div:nth-child(2) div[data-id]') or page_number > 5:
                print("End of Product listing...")
                break

            products_data = extract_products_info(html_content)
            all_products_data.extend(products_data)

            page_number += 1

    # Plot the Price/Rating graph
    plot_PR_graph(all_products_data)

if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Output Graph:

Image description

Drawing Insights from Scraped Data

From the visualization above, we observe that headphones with higher ratings tend to have a higher average price, suggesting that customers may be willing to pay a premium for better-rated products. However, it's essential to consider other factors, such as brand reputation, features, and customer reviews, before drawing conclusive insights.

In addition to the above insights, further analysis could include:

  • Correlation between 'Review Count' and 'Rating' to understand if highly-rated products also have more reviews.
  • A price distribution analysis was used to identify the most common price range for headphones on Flipkart.

By combining data scraping with visualization techniques, businesses and consumers can make more informed decisions and gain a deeper understanding of market dynamics.

Final Words

This guide has provided the necessary insights to scrape Flipkart products utilizing Python and the Crawlbase Crawling API. Should you further your expertise in extracting product information from additional e-commerce platforms such as Amazon, Walmart, eBay, and AliExpress, we encourage you to consult the supplementary guides provided.

Here are some other web scraping python guides you might want to look at:

📜 How to scrape Producthunt

📜 How to scrape Yelp

📜 How to scrape Images from DeviantArt

📜 How to scrape Github

📜 How to Build a Reddit Scraper

📜 Instagram Proxies to Scrape Instagram

We understand that web scraping can present challenges, and it's important that you feel supported. Therefore, if you require further guidance or encounter any obstacles, please do not hesitate to reach out. Our dedicated team is committed to assisting you throughout your web scraping endeavors.

Frequently Asked Questions

Q. Is it legal to scrape data from Flipkart?

Web scraping Flipkart, exists in a legal gray area. While the act of scraping itself might not be explicitly illegal, the use and dissemination of the scraped data can raise legal concerns. It's imperative to meticulously review Flipkart's terms of service and the directives in their robots.txt file. These documents often provide guidelines about permissible activities and data usage restrictions. Furthermore, scraping should not violate any copyright laws or infringe upon Flipkart's intellectual property rights. Before engaging in any scraping activities, it's wise to seek legal counsel to ensure compliance with local regulations and to mitigate potential legal risks.

Q. Why is scraping Flipkart search pages beneficial?

Scraping Flipkart's search pages provides businesses with valuable insights into the e-commerce landscape. Here's why it's advantageous:

  • Product Trends: Monitoring Flipkart's search data reveals emerging product trends, helping businesses align their offerings with market demands.
  • Pricing Intelligence: By analyzing product prices on Flipkart, businesses can refine their pricing strategies, ensuring competitiveness without compromising profitability.
  • Consumer Insights: Search patterns on Flipkart offer a glimpse into consumer behavior, guiding businesses in product development and marketing efforts.
  • Competitive Edge: Access to real-time data from Flipkart gives businesses a competitive advantage, enabling swift, informed decisions in a fast-paced market.

In summary, scraping Flipkart search pages equips businesses with actionable insights, fostering informed strategies and enhancing market responsiveness.

Q. How often should I update my Flipkart scraping script?

Flipkart, like many e-commerce platforms, undergoes frequent updates to enhance user experience, introduce new features, or modify its website structure. These changes can inadvertently disrupt your scraping process if your script is not regularly maintained. To maintain the integrity and efficiency of your scraping efforts, it's recommended to monitor Flipkart's website for any changes and adjust your script accordingly. Periodic reviews, perhaps on a monthly or quarterly basis, coupled with proactive script adjustments, can ensure that your data extraction remains accurate and uninterrupted.

Q. How can I handle potential IP bans or restrictions while scraping Flipkart?

Facing IP bans or restrictions is a common hurdle for web scrapers, especially when dealing with platforms as stringent as Flipkart. To navigate these challenges and ensure uninterrupted scraping:

  • Implement Delays: Introduce random or systematic delays between your scraping requests to mimic human behavior and reduce the load on the server.
  • Use Proxies: Use Rotating IP addresses to mask your IP address and distribute requests, making it harder for websites to track and block your scraping activity.
  • Rate Limiting Tools: Consider integrating middleware or tools designed to manage and respect rate limits, adjusting your scraping speed dynamically based on server responses.

Lastly, for those seeking a more structured and efficient approach, specialized services like Crawlbase's Crawling API for Flipkart can be invaluable. These platforms provide pre-optimized solutions designed to handle potential restrictions, offering a seamless and compliant scraping experience.

Image
Streamlining Proxy Integration in Clash for Windows

proxy
Image
The 10 Most Scariest Things About Repairing Bifold Doors
Image
FinTech Software Development Company: Your Partner for Innovation and Growth

fintech, softwaredevelopment, softwareengineering, bestofdev
Image
Illuminate Your Home with Brass Diyas for Pooja
Image
Sneaker Raffle Bots for Every Collector

sneakerbots, residentialproxy
Image
Kanapy Marzeń: Jak Wybrać Idealny Mebel do Twojego Salonu?
Image
Pure Component in React.js

webdev, javascript, beginners, programming
Image
The Importance of Food Donation in Delhi

donation, food, donate, fooddonation
Image
📝 30 Common PostgreSQL Interview Questions 🐘

webdev, postgres, interview, beginners
Image
Empower Your Internet Access with Proxy Configuration in Clash for Windows

proxy, clashforwindows
Image
Devops pipeline on a Golang Web App with Gitops and Kubernetes

cicd, kubernetes, go, githubactions
Image
How to Find Affordable Laser Hair Removal Options in Dubai

mentalhealth
Image
How Sneaker Raffle Bots Can Transform Your Collection

sneakerrafflebots, proxy
Image
Setting Development Goals That Actually Stick: Examples and Strategies
Image
Why Doesn't Postgres Use My Index

sql, database, postgres
Image
Adding a Rubocop config to an old repository | step-by-step guide

ruby, rails, tutorial, codequality
Image
Blur page body when hover on Nav Bar

codepen, webdev, css, bootstrap
Image
Stopping Bad Habits Before They Stop You
Image
Mastеring Usеr Accеptancе Tеsting (UAT) in Salеsforcе: A Comprehensive Guide

usеr, accеptancе, tеsting
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player