Detect Hallucinations with Ollama's Bespoke-Minicheck Model

n8n AI Research Productivity

Contributed by Sabrina Ramonov 🍄. From the community directory at agents.sabrina.dev, republished with permission and full credit.

Overview

This workflow is designed for automated fact-checking of texts. It uses AI models to compare a given text with a list of facts and identify potential discrepancies or hallucinations.

Components

1. Input

The workflow can be initiated in two ways:

a) Manually via the "When clicking 'Test workflow'" trigger

b) By calling from another workflow via the "When Executed by Another Workflow" triggerRequired inputs:

- facts: A list of verified facts

- text: The text to be checked

2. Text Preparation

The "Code" node splits the input text into individual sentences.

It takes into account date specifications and list elements.

3. Fact Checking

Each sentence is individually compared with the given facts.

It uses the "bespoke-minicheck" Ollama model for verification.

The model responds with "Yes" or "No" for each sentence.

4. Filtering and Aggregation

Sentences marked as "No" (not fact-based) are filtered.

The filtered results are aggregated.

5. Summary

A larger language model (Qwen2.5) creates a summary of the results.

The summary contains:

- Number of incorrect factual statements

- List of incorrect statements

- Final assessment of the article's accuracyUsage

Ensure the "bespoke-minicheck" model is installed in Ollama (ollama pull bespoke-minicheck).

Prepare a list of verified facts.

Enter the text to be checked.

Start the workflow.

The results are output as a structured summary.

Notes

The workflow ignores small talk and focuses on verifiable factual statements.

Accuracy depends on the quality of the provided facts and the performance of the AI models.Customization Options

The summarization function can be adjusted or removed to return only the raw data of the issues found.

The AI models used can be exchanged if needed.

This workflow provides an efficient method for automated fact-checking and can be easily integrated into larger systems or editorial workflows.

Tools used: Ollama, AI Agent

Download the workflow template (JSON)

Original tutorial

Want this running in your business? KOBA42 builds and operates automations like this one.