Skip to content
NEWError doctor and Watchtower: failed n8n runs explained in plain words, and emailed to you. See what is new

Library / Web Scraping & Research

Web Scraping & Research

Crawl Company Websites Autonomously with OpenAI and Supabase

When you run it, it takes a list of company sites, has OpenAI navigate and extract key information and stores the results in Supabase.

Import it free Sign in to open it

What it does

## Autonomous AI crawler This workflow autonomously navigates through given websites and retrieves social media profile links. 💡 You can modify this workflow to retrieve other type of data (e.g. contact details or company profile summary).

The steps

  1. Text
  2. URLs
  3. OpenAI Chat Model
  4. JSON Parser
  5. Map company name and website
  6. Execute workflow
  7. Get companies
  8. Select company name and website
  9. Set social media array
  10. Merge all data
  11. Insert new row
  12. Convert HTML to Markdown
  13. Retrieve URLs
  14. Split out URLs

28 steps in total.

Similar automations

Web Scraping & ResearchAutonomous AI Crawler for Company Websites with Supabase28 stepsLead Generation & EnrichmentExtract Company Names and Websites into Supabase48 stepsLead Generation & EnrichmentExtract Company Names and Websites into Supabase48 stepsWeb Scraping & ResearchCrawl a URL and Return Metadata with Serper3 steps