Skip to content
NEWMarketplace and the trig8n Agent: sell what you build, and get plans and changes that are tested before you see them. See what is new

Library / Web Scraping & Research

Web Scraping & Research

Extract Structured Web Data with Bright Data and Gemini

When you run it, the workflow scrapes a page to markdown with Bright Data and uses Google Gemini to extract structured data, notifying a webhook.

What it does

# Structured Data Extract, Data Mining with Bright Data & Google Gemini ## Overview Automated workflow: Structured Data Extract, Data Mining with Bright Data & Google Gemini. This workflow integrates 10 different services: stickyNote, httpRequest, lmChatGoogleGemini, readWriteFile, chainLlm. It contains 31 nodes and follows best practices for error handling and security. ## Workflow Details - Total Nodes: 31 - Node Types: 10 - Error Handling: ✅ Implemented - Security: ✅ Hardened (no sensitive data) - Documentation: ✅ Complete ## Node Breakdown - When clicking ‘Test workflow’: manualTrigger - Sticky Note: stickyNote - Sticky Note1: stickyNote - Markdown to Textual Data Extractor: chainLlm - Set URL and Bright Data Zone: set - Initiate a Webhook Notification for Markdown to Textual Data Extraction: httpRequest - Initiate a Webhook Notification for AI Sentiment Analyzer: httpRequest - Google Gemini Chat Model for Data Extract: lmChatGoogleGemini - Google Gemini Chat Model for Sentiment Analyzer: lmChatGoogleGemini - Perform Bright Data Web Request: httpRequest - ... and 21 more nodes ## Usage Instructions 1. Configure Credentials: Set up all required API keys and credentials 2. Update Variables: Replace any placeholder values with actual data 3. Test Workflow: Run in test mode to verify functionality 4. Deploy: Activate the workflow for production use ## Security Notes - All sensitive data has been removed or replaced with placeholders - Error handling is implemented for reliability - Follow security best practices when configuring credentials ## Troubleshooting - Check error logs if workflow fails - Verify all credentials are properly configured - Ensure all required services are accessible

The steps

  1. When clicking ‘Test workflow’
  2. Markdown to Textual Data Extractor
  3. Set URL and Bright Data Zone
  4. Initiate a Webhook Notification for Markdown to Textual Data
  5. Initiate a Webhook Notification for AI Sentiment Analyzer
  6. Google Gemini Chat Model for Data Extract
  7. Google Gemini Chat Model for Sentiment Analyzer
  8. Perform Bright Data Web Request
  9. Topic Extractor with the structured response
  10. Trends by location and category with the structured response
  11. Google Gemini Chat Model
  12. Initiate a Webhook Notification for trends by location and c
  13. Create a binary file for topics
  14. Write the topics file to disk

58 steps in total.

Similar automations