Many AI vendor demos have the same shape: a futuristic logo, a phrase like "powered by advanced AI," and a representative who sounds confident without saying anything testable. Nobody has to be lying for the buyer to leave without a clear answer. Small business owners are busy running the actual business and may not have an engineer in the room to ask the annoying follow-up question. So here are the follow-up questions. Ask them out loud in the meeting and listen for specific answers.
Start with whether the thing is actually looking at your data or just performing on a script. A lot of "AI-powered" demos are really just a well-rehearsed walkthrough against a fixed dataset the vendor built specifically to look good. That's not fraud, it's just not proof of anything about your business. Ask them to run it live, in the room, against your actual product catalog, your actual customer records, your actual call transcripts. If the answer is "we'll need to schedule a custom onboarding for that" before you can see it work on your own information, that's not a no, but it's a flag to keep in your pocket. A tool that's genuinely connected to your systems should be able to show you something real, even if it's rough, faster than a tool that's just replaying a highlight reel.
Next, ask directly what happens to your data once it goes into their system. Is it stored, and for how long? Is it used to train or improve their model, and if so, does that mean it could influence or resemble what shows up for other customers, including possibly your competitors? Can you get it deleted on request, and is that in writing anywhere, or is it a verbal assurance from a rep who won't be in that job in a year? You want a real answer, ideally a link to an actual data processing addendum, not a reassuring shrug. If a vendor gets cagey about where your customer records or your internal documents end up, that's worth more weight than almost anything else in the pitch, because that's the part that can actually hurt you later.
Then ask what model is actually running under the hood. Not the marketing name, the real one. "Our proprietary AI engine" often means a wrapper around a general-purpose model from one of the handful of companies that actually build these things, with prompt engineering and business logic layered on top. That is a completely legitimate way to build a product, and many useful tools work exactly that way. But you should know which model it is, roughly how it is used, and what happens to your product when that underlying model is upgraded, deprecated, or changed. If a vendor treats the model as a trade secret it cannot discuss at all, be cautious: secrecy can hide how little product-specific value has been built around it.
Cost is where vague claims can do the most damage, because the pricing in the pitch deck is rarely the pricing you will actually pay. Ask what the product costs at your real volume, not the volume in the example slide. Many tools are priced per seat, or with a headline number that only works if you barely use the product, then cross into metered usage, API charges, or transaction fees when adopted into daily operations. Ask the vendor to run the math with your actual numbers during the call. If the answer keeps gaining caveats, budget around the worse case rather than the pitch case.
Ask whether anyone outside the company has actually verified what they're claiming. A case study on their own website is marketing, not evidence, even when it has a logo and a quote attached. What you want is permission to call an actual current customer directly, unsupervised, and ask them how it's really going. Reputable vendors can usually produce at least one or two references who'll take that call. If every reference is filtered through the vendor's own team, or the case studies are all anonymized ("a leading retailer saw a 40% improvement"), you're not being shown proof, you're being shown a story, and there's a difference between the two that matters a lot once you've signed a year-long contract.
The clearest red flags tend to cluster together, and they are worth naming plainly. Vague or shifting answers to direct technical questions. A refusal to explain, even at a basic level, what happens when you send the product data. Pricing that only pencils out at an unrealistic usage level, paired with reluctance to model your real numbers. Testimonials with no name attached. A demo that cannot be run live against anything real. None of these alone means walk away; some good vendors are simply poor at explaining themselves under pressure. But two or three in the same pitch form a pattern, and patterns are more useful than any single answer.
Bring these questions into the actual sales call rather than saving them for a follow-up email, because the live reaction tells you more than the polished written response. A live answer about data retention or cost at real volume shows whether the person across the table understands the product or is simply presenting it. Then get anything that matters, especially around data handling and pricing at scale, in writing before you sign. A verbal assurance from a sales representative is worth very little once you are a support ticket instead of a prospect.
None of this is about being anti-AI or assuming every vendor pitching you is dishonest, because most aren't. It's that the AI vendor market right now rewards confident vagueness, and a business owner without a technical background is exactly the audience that vagueness works best on. Asking pointed, specific questions doesn't make you difficult, it makes you a customer worth taking seriously, and the vendors worth working with will actually respect it. The ones who get defensive or evasive when you ask are telling you something true about how the rest of the relationship will go.
I write about this kind of thing regularly at 013labs.com, mostly because I'd rather a business owner spend twenty minutes reading a skeptic's checklist than twenty thousand dollars finding out the hard way.