How ChatGPT filters Shopify catalog for a shopper's query

When a shopper types a sentence into ChatGPT, it turns that sentence into a query and calls Shopify's Catalog API. That's the same call an agent makes against your store.

I wanted to see exactly what that call does with a real query, so I built queries one word at a time and ran each version through the API.

1/ dresses - ten results come back, all dresses. The category word does its job.

2/ black dresses - the second result was a white dress. The third ranks on the word "black" sitting in its title, not on an actual color field. Color is filterable in the Catalog API. Nobody set it, so the query fell back to matching the word instead.

3/ black dresses for an evening party - this should have been the hardest cut of the three. Nothing dropped. The order shuffled and the black dress I expected to win slid to eighth, behind the white one. Occasion has no filter, so it never got the chance.

You can run the test here → Shopify Catalog Audit tool

The filter list is short

The list Shopify's Catalog API can filter right now is short, no matter how much product data you have - it’s price, rating, availability, condition, category, shipping. Plus one attribute filter that only takes three values: color, size, target gender.

Everything else - occasion, fit, fabric, material - still gets read. It gets scored as text sitting next to the rest of the query, same way "evening party" above did with the dresses query, instead of being used to remove anything.

Filterable means the API can cut a result in or out because of it. Readable means the words are there to match against. A merchant can fill in an occasion field and it won't drop a single wrong result, because it was never wired to a filter to begin with.

When the field is just empty

So filters are one kind of gap, it exists no matter what a merchant does. There's a second kind and it only shows up when the data itself is missing.

Some fields genuinely are filterable today - a price, a material, a size.

Separately, I tested five ways a query typically narrows down as a shopper gets more specific, like stacking several attributes at once, a number or a spec, ruling something out, a price cap, and whether something fits a car.

Out of all, only stacking attributes does well. Retrieval here is semantic, so a query naming several properties leans toward the products that already carry all of them.

The rest are comparisons, and a comparison needs a value on the other side. For example, under $250 has to check a number against a number and something like not made of linen has to check a material against a material.

When that field is filled in, the API filters on it correctly. When it's blank, there's nothing to compare against, so the whole constraint gets scored as text like every other word in the query. A $400 product can land in a shortlist that should have been capped. A linen dress can come back for someone who writes “not linen”.

The same pattern, at scale

The same five query shapes turn up again in the Catalog Visibility Report. We tested this independently across 12 categories on a much bigger sample.

Rule out linen on a pair of summer trousers and 8 of 10 results come back in linen anyway. Cap a pair of jeans at $90 and 4 of 10 come back over budget. Ask for a $20 bottle of olive oil and only 1 of 10 holds to it.

Whatever field the query needs, if it isn't a stated value on the product, the comparison can't run. Check out the full Catalog Visibility Report →

What this means for a Shopify product listing

Check whether the attribute your customers search by is even something Shopify can filter on. That's color, size, and gender, until the API opens up more.

Then check if the fields that are filterable have been filled in your product data.

Neither fix is about ChatGPT specifically. Catalog API is what Google AI Mode and every other agent reads from too, so the same gap shows up everywhere at once.

Run the audit again → see what your catalog is missing and what’s inferred versus filterable. Check how you perform against your competitors in the query tests.

- Ankit