Automate SEO Site Analysis with Google, ScrapingNinja, and OpenAI

Scenario: SEO Site Analysis Bot

Overview:

The SEO Site Analysis Bot automates the process of analyzing e-commerce websites to evaluate their SEO performance, persuasive language, and overall strategies. It connects to Google Custom Search, ScrapingNinja API, and OpenAI's GPT-4 to fetch and analyze site data, then compiles a detailed report and customer persona, storing the results in Google Docs.

Flow Description:

1. Set Shopify Name Variable

• ID: 58
• Module: util:SetVariable2
• Version: 1
• Description: Sets the Shopify store URL variable.
• Parameters: {}
• Mapper:
• Name: "Shopify name"
• Scope: "roundtrip"
• Value: "https://quicklifesolutions.com"
• Metadata:
• Designer: { "x": 0, "y": 0 }

1. Set API Keys Variables

• ID: 59
• Module: util:SetVariables
• Version: 1
• Description: Sets the API keys for Google and RapidAPI.
• Parameters: {}
• Mapper:
• Scope: "roundtrip"
• Variables:
• Google Api Key: "AIzaSyCzCkAMTuVJmsrrR4H4pFt7OEsxMuYA-zo"
• Search Engine ID: "05aaae459fd564f87"
• Rapid Api Key: "af804cf607msh955f19a9c0775a4p1c08a8jsn6095f28eecd4"
• Metadata:
• Designer: { "x": 300, "y": 0 }

1. Call Custom Search API

• ID: 2
• Module: http:ActionSendData
• Version: 3
• Description: Calls the Google Custom Search API to search for the Shopify store.
• Parameters:
• Handle Errors: false
• Use New ZLib DeCompress: true
• Mapper:
• URL: "https://www.googleapis.com/customsearch/v1?key={{59.Google Api Key}}&cx={{59.Search Engine ID}}&q={{58.Shopify name}}"
• Method: "get"
• Gzip: true
• Parse Response: true
• Follow Redirect: true
• Reject Unauthorized: true
• Metadata:
• Designer: { "x": 600, "y": 0, "name": "Call Custom Search API" }

1. Basic Router

• ID: 4
• Module: builtin:BasicRouter
• Version: 1
• Description: Routes the flow based on the API response.
• Mapper: null
• Metadata:
• Designer: { "x": 900, "y": 0 }

1. ScrapingNinja API Call

• ID: 9
• Module: http:ActionSendData
• Version: 3
• Description: Calls the ScrapingNinja API to scrape the top-ranking e-commerce site.
• Parameters:
• Handle Errors: false
• Use New ZLib DeCompress: true
• Mapper:
• URL: "https://scrapeninja.p.rapidapi.com/scrape"
• Method: "post"
• Gzip: true
• Headers:
• {"name": "content-type", "value": "application/json"}
• {"name": "X-RapidAPI-Key", "value": "{{59.Rapid Api Key}}"}
• {"name": "X-RapidAPI-Host", "value": "scrapeninja.p.rapidapi.com"}
• Data: "{\n\n \"url\": \"{{2.data.items[].link}}\"\n\n}"
• Parse Response: true
• Follow Redirect: true
• Reject Unauthorized: true
• Metadata:
• Designer: { "x": 1200, "y": 0, "name": "ScrapingNinja API Call" }

1. HTML to Text Conversion

• ID: 21
• Module: regexp:HTMLToText
• Version: 1
• Description: Converts the scraped HTML content to plain text.
• Parameters: {}
• Mapper:
• HTML: "{{9.data.body}}"
• Newline: "lf"
• Uppercase Headings: true
• Metadata:
• Designer: { "x": 1500, "y": 0 }

1. Remove Image URLs

• ID: 36
• Module: regexp:Replace
• Version: 1
• Description: Removes image URLs from the text to reduce character count.
• Parameters: {}
• Mapper:
• Text: "{{21.text}}"
• Pattern: "cdn\.shopify\.comsfiles.*?\.(jpg|png)"
• Value: "{{emptystring}}"
• Global: true
• Metadata:
• Designer: { "x": 1800, "y": 0 }

1. Set Variable for Site Text

• ID: 38
• Module: util:SetVariable2
• Version: 1
• Description: Saves the cleaned site text into a variable.
• Parameters: {}
• Mapper:
• Name: "site_3"
• Scope: "roundtrip"
• Value: "{{split(36.text; newline)}}"
• Metadata:
• Designer: { "x": 2100, "y": 0 }

1. Deduplicate Text

• ID: 40
• Module: util:SetVariable2
• Version: 1
• Description: Removes duplicate lines from the site text.
• Parameters: {}
• Mapper:
• Name: "site_3"
• Scope: "roundtrip"
• Value: "{{deduplicate(38.site_3)}}"
• Metadata:
• Designer: { "x": 2400, "y": 0 }

1. Aggregate Text

• ID: 42
• Module: util:TextAggregator
• Version: 1
• Description: Aggregates the cleaned and deduplicated text into a single block.
• Parameters: {}
• Mapper:
• Value: "{{40.site_3}}"
• Metadata:
• Designer: { "x": 2700, "y": 0 }

1. Analyze Site with GPT-4

• ID: 51
• Module: openai-gpt-3:CreateCompletion
• Version: 1
• Description: Uses GPT-4 to analyze the e-commerce site and provide a detailed report.
• Parameters:
• Connection: 531539
• Mapper:
• Model: "gpt-4"
• Select: "chat"
• Messages:
• {"role": "system", "content": "You are a website analyst. You analyze e-commerce website home pages to determine what they are doing well, what they are doing poorly, and how a competitor may emulate what is working while avoiding what is not."}
• {"role": "user", "content": "Analyze the e-commerce site (top ranking on Google) for {{58.Shopify name}} products below. Then provide a report including - what elements of the page are set up using SEO best-practices? What persuasive language is being used that may lead to higher conversions? What strategies can you identify that may be contributing to high rank on Google? What aspects of the site could use improvement?\nE-commerce site - {{substring(trim(replace(replace(replace(replace(replace(replace(42.text; \"*\"; emptystring); \"/\"; emptystring); \"\\"; emptystring); \"+\"; emptystring); \"[\"; emptystring); \"]\"; emptystring)); 0; 21000)}}"}
• Max Tokens: "1000"
• Temperature: "1.1"
• Metadata:
• Designer: { "x": 3000, "y": 0, "name": "Analyze Site" }

1. Set Analysis Result Variable

• ID: 54
• Module: util:SetVariable2
• Version: 1
• Description: Saves the analysis result into a variable.
• Parameters: {}
• Mapper:
• Name: "site_3"
• Scope: "roundtrip"
• Value: "{{trim(replace(51.choices[].message.content; \"\"\"\"; emptystring))}}"
• Metadata:
• Designer: { "x": 3300, "y": 0

1. }
2. Get Analysis Result Variable

• ID: 55
• Module: util:GetVariables
• Version: 1
• Description: Retrieves the analysis result variable for further processing.
• Parameters: {}
• Mapper:
• Variables: ["site_3"]
• Metadata:
• Designer: { "x": 3600, "y": 0 }

1. Create Customer Persona with GPT-4

• ID: 16
• Module: openai-gpt-3:CreateCompletion
• Version: 1
• Description: Uses GPT-4 to create a customer persona based on the site analysis.
• Parameters:
• Connection: 531539
• Mapper:
• Model: "gpt-4"
• Select: "chat"
• Messages:
• {"role": "system", "content": "You are a website analyst. You analyze e-commerce website home pages to determine what they are doing well, what they are doing poorly, and how a competitor may emulate what is working while avoiding what is not."}
• {"role": "user", "content": "Based on the following site analyses, create a customer persona that fits the niche, products, and offerings of these sites (the sites were pulled from Google for the keyword {{58.Shopify name}}).\nSite analyses:\n3/ {{55.site_3}}"}
• Max Tokens: "1000"
• Temperature: "1.1"
• Metadata:
• Designer: { "x": 3900, "y": 0, "name": "Customer Persona Creator" }

1. Create Google Doc

• ID: 57
• Module: google-docs:createADocument
• Version: 1
• Description: Creates a Google Doc with the site analysis and customer persona.
• Parameters:
• Connection: 531454
• Mapper:
• Name: "Website Analysis & Customer Persona ({{58.Shopify name}})"
• Footer: false
• Header: false
• Content: "Site 1 - {{55.site_3}}\n\nCustomer Persona - {{16.choices[].message.content}}"
• FolderId: "/"
• Destination: "drive"
• Metadata:
• Designer: { "x": 4200, "y": 0 }

Technical Details:

• Built-in Modules: Basic Router
• On-error Modules: None specified
• App Connections: Google Custom Search, ScrapingNinja API, OpenAI (GPT-4), Google Docs

This scenario automates the process of analyzing e-commerce websites for SEO performance, persuasive language, and overall strategies. It uses Google Custom Search to find relevant sites, ScrapingNinja to scrape content, and GPT-4 to analyze the data and create detailed reports and customer personas, which are then saved to Google Docs.