n8n AI Data Extraction Internet of Things
Contributed by Sabrina Ramonov 🍄. From the community directory at agents.sabrina.dev, republished with permission and full credit.
This workflow with AI agent is designed to navigate through the page to retrieve a specific type of information (in this example: social media profile links).
The agent is equipped with two tools:
- Text tool: to retrieve all the text from the page.
- URLs tool: to extract all possible links from the page.
You can edit the prompt and JSON schema connected to the agent in order to return other data than social media profile links.
This workflow uses Supabase as storage (input/output). Feel free to change it to any other database of your choice.
See this workflow in action in my YouTube video.
How it works?
The workflow uses the input URL (website) as a starting point to retrieve the data (e.g. example.com). Using the "URLs tool," the agent is able to retrieve all links from the page and navigate to them.
For example, if you want to retrieve contact information, the agent will try to find a subpage that might contain this information (e.g. example.com/contact) and extract the information using the text tool.
Set up steps:
1. Connect database with input data (website addresses) or pin sample data to trigger node.
2. Configure the crawling agent to retrieve the desired data (e.g. modify prompt and/or parsing schema).
3. Set credentials for OpenAI.
4. Optionally: split agent tools to separate workflows.
If you like this workflow, please subscribe to my YouTube channel and/or my newsletter.
Tools used: OpenAI, Supabase, AI Agent
Download the workflow template (JSON)
Want this running in your business? KOBA42 builds and operates automations like this one.