You may have seen my recent video about whether or not keywords are dead. They’re not, and neither is tracking them in order to benchmark and monitor SEO performance. It all makes for nice clickbaity articles, but the matter of the fact that they are still very much alive.

There are many excellent SEO rank tracking tools available, I have used Advanced Web Ranking (AWR) since i started doing SEO in 2010 and it’s features from classic keyword tracking and gap analysis to market share and monitoring search intent are such that i’ll probably continue to use this for several years to come. Especially as the price point is excellent.

That said and as I have always said, good SEOs should get their data from a range of sources before using their own intution in terms of what this tells us. With this in mind, services like SerpAPI and DataforSEO stand out as more sophisticated rank tracking solutions due to their cost-effectiveness and the level of detailed data they provide.

To newbies or more content focussed SEOs, these tools might seem a bit technical at first, especially if you’re not familiar with APIs or programming. However, with a bit of guidance, you can harness the power of these tools to gain deep insights into your SERP performance.

This guide will walk you through the process of using SerpAPI and DataforSEO, which was shared by Chris Green a few weeks ago on LinkedIn. This guide talks you through testing them out in their playground environments and implementing them via Google Sheets or custom workflows.

Understanding SerpAPI and DataforSEO

SerpAPI and DataforSEO are APIs that allow you to scrape and collect data from SERPs. They enable you to pull detailed information about search results, including rankings, URLs, snippets, ads, and more, from different search engines and locations.

• SerpAPI offers an easy-to-use interface and comprehensive documentation, making it accessible even for beginners.
• DataforSEO is known for its robust features and is widely used by SEO professionals to gather large amounts of data efficiently.

These tools allow you to track your website’s performance on Google, and other search engine should you so wish.

Step By Step Guide to Using SerpAPI and DataforSEO

Explore the Tools

Visit the Playgrounds

• To get started, familiarise yourself with the tools by using their “playground” functionalities. The playgrounds allow you to test out the APIs in a user-friendly environment without needing to write any code.
• SerpAPI Playground
• DataforSEO Playground
• In these playgrounds, you can simulate API requests to see the type of data you can retrieve, such as search results for different queries, locations, and devices.

Understand the Data

• Take some time to explore the types of data these tools can extract. This includes organic search results, paid ads, local pack results and more.
• Experiment with different parameters like location, device type, and search engine to see how they affect the data.

Test the Tools

Practice Scraping

• Use the playgrounds to practice scraping SERPs. Start with basic queries and gradually try more complex ones, adjusting parameters such as location or device type to see how the results change.
• The playgrounds will give you a feel for how the API requests work and the kind of information you can obtain.

Evaluate the Data

• Analyse the data you retrieve to understand how it can be useful for your SEO efforts. Look for patterns such as how rankings vary by location or device and consider how you can use this information to inform your SEO strategies.

Decide Your Approach

Google Sheets

• If you prefer to avoid coding, both SerpAPI and DataforSEO offer integrations with Google Sheets. This allows you to pull SERP data directly into a spreadsheet, making it easier to analyse and share.
• Set up your account with SerpAPI or DataforSEO and follow the Google Sheets integration guides:
  • SerpAPI Google Sheets Guide
  • DataforSEO Google Sheets Guide
• These guides will walk you through the process of connecting the API to your Google Sheets, enabling you to automate data retrieval without writing any code.

Setting Up Your Spreadsheet

• Once integrated, you can set up your spreadsheet to regularly pull in new SERP data. This can be particularly useful for tracking changes over time, monitoring your competitors or analysing how your rankings vary across different search engines or regions.

Build Custom Workflows

For Coders and Advanced Users

• If you’re comfortable with programming you can create custom workflows using languages like Python, PHP, or Bash. This allows you to automate data extraction, customise your requests and integrate the data into your existing systems.

Making API Requests

• Write scripts to make API requests to SerpAPI or DataforSEO. The APIs typically return data in JSON format which you can then parse and manipulate according to your needs.

Python Example

import requests

url = "https://serpapi.com/search"
params = {
    "engine": "google",
    "q": "your query",
    "location": "New York, NY",
    "api_key": "your_api_key"
}

response = requests.get(url, params=params)
data = response.json()

Extracting and Storing Data

• After retrieving the data, you can extract specific elements and store them in a format that suits your needs such as a CSV file or a database.

Python Example

import pandas as pd

# Assuming `data` is the JSON response from the API
results = data['organic_results']
df = pd.DataFrame(results)
df.to_csv('serp_data.csv', index=False)

Automating Your Workflow

• You can automate this process to run at regular intervals, ensuring that you always have up-to-date data without manual intervention.
• Consider setting up cron jobs or similar automated tasks to regularly fetch and process the data.

Take It One Stage at a Time

Start Basic

If you’re new to APIs or data extraction, start with the basics. Use the playgrounds and Google Sheets integrations to get comfortable with the process before moving on to more advanced techniques.

Learn and Iterate

As you gain confidence, explore more complex queries and workflows. The more you experiment, the better you’ll understand how to leverage this data for your SEO efforts.

Continuous Improvement

Stay updated with the latest features and updates from SerpAPI and DataforSEO. These platforms regularly add new functionalities that can further enhance your data collection and analysis processes.

Summary

Using SerpAPI and DataforSEO allows you to track and analyse SERP data, in near real time and mean you don’t have to waste valuable credits with your enterprise level rank tracking software.

Whether you choose to use Google Sheets for a no-code solution or dive into custom workflows with programming, these tools offer flexibility and power at a cost-effective price.

It may then be that you decide to use this kind of method for your client facing rank tracking and reporting, instead of the more aforementioned tools.

But just be wary that this doesn’t come with some of the bells and whistles i’ve mentioned and a nice interface that you get with the likes of AWR and SERanking – but for the price I believe this is well worth experimenting with nonetheless and can complement your existing stack.