Your Shopify store is probably telling AI the wrong things about your business
Shopify builds a set of default answers about your store from your settings and hands them to AI agents whenever they ask. For a lot of merchants, those answers describe a business they don't run.
We ran into this on a store we were setting up. It doesn't sell anything: no products, no cart, no checkout. We asked its agent endpoint whether it ships internationally and got back a confident list of 28 countries.
Nobody had typed that in. Shopify worked it out from the shipping zones sitting in settings, which were only there because every Shopify store gets them by default. An AI agent asking about that business would have walked away thinking it was a store that ships worldwide.
Where do these answers come from?
Shopify's Knowledge Base creates a starter set of FAQs from your existing setup. The usual defaults cover:
- What countries you ship to, from your delivery profiles
- Whether you offer local pickup, from your locations
- Whether customers can manage their own returns, from customer accounts
- Whether you accept store credit
- What languages your store is available in
Each one is technically true about your settings. Whether it's true about your business is a different matter, and the system can't tell the difference.
Why does this matter more than a normal content error?
Because of who's reading it and how. A person browsing your site brings context with them. They can see you're an agency, a wholesaler or a made-to-order studio, and they'll ignore a stray line about shipping.
An AI agent doesn't have that context. It queries a structured endpoint, gets a definite answer, and repeats it as fact, with no hedging and no visible source for the person asking. And an empty answer and a wrong answer aren't equally bad. Silence comes across as uncertainty, but a confident wrong answer comes across as information.
The defaults aren't neutral. They describe a generic product store, get applied to your brand, and are served to systems that won't question them.
How do I check what my store is saying?
You don't need any special tools. Every Shopify store has an endpoint that AI agents query. Send a plain-English question to yourstore.com/api/mcp using the search_shop_policies_and_faqs tool, and you'll get back exactly what an agent would. If your products aren't showing up either, our post on showing up in ChatGPT covers the product side.
Ask it these five things:
- What does [your brand] do?
- Do you offer [your main service or product category]?
- Do you ship internationally?
- What's your return policy?
- How do I contact you?
Watch out for two kinds of failure. An empty result means the agent has nothing to say about you at all. An answer pulled from settings you never thought of as public means it may be describing a business you don't run.
On the store we audited, the first two questions came back empty and the third returned that shipping list. The most important question a buyer could ask, what does this company do, had no answer.
How do I fix it?
You add your own question-and-answer pairs, and those take priority over the generated ones. In the Shopify admin, you'll find this under Apps, then Knowledge Base, then Add FAQ. Our Knowledge Base setup guide covers loading them in bulk.
Three things to know before you start.
Write the corrections, not just the additions. It's tempting to add what's missing and leave the defaults alone. If a default is wrong, write an FAQ that answers the same question correctly. On the store we fixed, adding a simple answer to "what countries do you ship to" replaced the 28-country list completely.
Answer the questions people really ask. Matching is based on meaning, so phrasing matters. "Is there a minimum revenue?" and "are we too small for you?" are the same question to a person but different questions to a retrieval system. Cover both.
Be careful with numbers. This one tripped us up. The matcher compares meaning, not math. We wrote separate answers for different revenue tiers and ended up telling a brand doing $8k a month that they were comfortably in range, because their wording happened to sit closest to the wrong entry. The fix was one answer that states the rule and both sides of it, so whichever entry matches, the reader still gets the right information.
What should I do first?
Query the endpoint and write down what comes back. It takes ten minutes, and it's the only way to know whether this applies to you.
If everything's accurate, you're done and it only cost you ten minutes. If not, you've found something that's been answering on your behalf, to an audience you can't see, for as long as the feature has existed.
Setting up what AI agents can say about a store is part of our AI implementation work. Want us to run that check as part of a wider review? It's included in our free CRO audit.