n8n AI Content Creation Engineering
Contributed by Sabrina Ramonov 🍄. From the community directory at agents.sabrina.dev, republished with permission and full credit.
This n8n template takes a video and extracts frames from it, which are used with a multimodal LLM to generate a script. The script is then passed to the same multimodal LLM to generate a voiceover clip.
This template was inspired by processing and narrating a video with GPT's visual capabilities and the TTS API.
How it works:
1. Video is downloaded using the HTTP node.
2. A Python code node is used to extract the frames using OpenCV.
3. A loop node is used to batch the frames for the LLM to generate partial scripts.
4. All partial scripts are combined to form the full script, which is then sent to OpenAI to generate audio from it.
5. The finished voiceover clip is uploaded to Google Drive.
Requirements:
- OpenAI for LLM
- Ideally, a mid-range (16GB RAM) machine for acceptable performance!
Customizing this workflow:
- For larger videos, consider splitting them into smaller clips for better performance.
- Use a multimodal LLM that supports video fully, such as Google's Gemini.
Tools used: OpenAI, Google Drive, Google Gemini
Download the workflow template (JSON)
Want this running in your business? KOBA42 builds and operates automations like this one.