Prompt AI Strategy Business Intelligence
Written and maintained by KOBA42. A free original, use it in any chatbot.
Paste an AI vendor's pitch or your demo notes and get the 15 questions that separate a real system from a good demo: who owns the code, how data is handled, what breaks at scale, lock-in, and the maintenance model. For anyone evaluating an AI vendor.
A vendor demo is engineered to answer the questions you would think to ask. This flips it: it reads their own claims back and generates the questions demos are built to dodge, especially ownership, what is shipping today versus roadmap, and who eats the maintenance cost later. The paired 'good answer / bad answer' tells let you grade responses live instead of realizing weeks later that you got hand-waved.
How to use it. Paste the vendor's pitch, page copy, or your demo notes where marked. You get 15 targeted questions in five groups, each with a tell for a good versus evasive answer, plus the two most load-bearing unverifiable claims and the question to open with. Bring it to the next call.
Worked example. Fed a pitch promising 'fully autonomous agents,' it produced questions like 'which actions run without a human approving them today, and which need a person in the loop.' On the call the vendor admitted three of the four flagship actions were human-approved, which reframed the entire pricing conversation.
If you want a second set of eyes on a shortlist before you sign, a vendor-fit review at koba42.com/contact is a fast way to pressure-test the finalists.
You are a vendor claim interrogator. I will paste an AI vendor's pitch, landing-page copy, or demo notes. Convert their claims into the specific questions that separate a real system from a good demo. Be skeptical but fair: you are not trying to embarrass anyone, you are protecting the buyer. [PASTE VENDOR PITCH OR DEMO NOTES HERE] Produce exactly 15 questions, grouped under these five headings, each phrased so that a vague answer is obvious: 1. Ownership and lock-in: who owns the code, prompts, and any fine-tuned models if we leave, and what exactly we can export. 2. Data handling: where our data goes, whether it trains their models, retention, and sub-processors. 3. Real vs demo: which parts of what I was shown ship today, which are roadmap, and which were hand-built for the demo. 4. Behavior at scale: what breaks or slows at 10x and 100x our volume, and what they have actually run at. 5. Maintenance model: who maintains prompts and integrations as things drift, who pays when a model deprecates, and their track record on this. For each question add a one-line "good answer looks like" and "bad answer looks like" so I can grade responses in the room. After the 15, list the two claims in their pitch that are least verifiable and most load-bearing, and name the single question I should open with. Do not invent claims they did not make. If the pitch is too thin to interrogate a heading, say what information is missing instead of padding.
Tools used: Claude, ChatGPT, Any LLM
Want this running in your business? KOBA42 builds and operates automations like this one.