Your AI Shopper Gets It Wrong And Buys Anyway

The robots are learning to shop for us, and right now they are confident, quick and often wrong

On November 5, Visa published the results of a new Agentic Commerce Benchmark, one of the first public tests of how well AI assistants can shop on someone's behalf. It ran 22 AI models from 12 different labs through 15 shopping challenges, the kind of tasks a busy person might hand to an assistant: find this item, in this size, under this price, from a store that ships in time.

The results were sobering. Across all the attempts, the assistants failed 42.8 percent of the time. The best model passed 82.2 percent of its tasks, and the weakest managed only 28.9 percent. Most striking of all, 54 percent of the failed attempts still ended in a purchase. The assistant did the wrong thing and bought something anyway. In everyday terms, it finished the shopping trip with the wrong bag.

The tasks were not exotic. They included choosing the right size, staying under a set budget and picking a store that could deliver by a certain date. These are the details a human shopper checks without thinking, and they are exactly where the assistants slipped, often by choosing something close enough and moving on as if the job were done.

Visa was careful to note what it had not yet tested. The benchmark stopped short of the final checkout and payment step, which the company plans to examine next. Even so, the picture is clear: the software is eager to finish the job, whether or not it has understood the job.

Shoppers Are Not Ready To Let Go

People seem to sense the risk. In a survey of 2,000 American adults conducted with Kantar in late September, the payments company FIS found that half want to approve every purchase an AI assistant makes for them. The most popular condition for trusting an assistant at all, chosen by 58 percent, was a guarantee of a refund when it makes a mistake.

That caution has not slowed the money. AI tools are rapidly becoming a place where shopping starts. Adobe has forecast that traffic arriving at retail sites from AI assistants will grow by 130 percent this holiday season. OpenAI has said its new advertising business inside ChatGPT has passed $1 billion in annualized revenue, which means brands are already paying to appear in answers.

Retailers are adjusting quickly. Some are building their own assistants, trained only on their catalogs, so customers can ask questions without leaving the store's website. Others are opening their product data to the big AI platforms in the hope of being recommended more often. Both approaches share one goal: making sure the assistant has the right facts before it answers.

So the market is moving fast in two directions at once. Shoppers want control, and companies are racing to build assistants that need less of it. For now the gap between those two is filled with errors, returns and confused customers. The brands that get ahead are preparing for both at once, today.

Who The Machines Believe

The most useful finding for brands came from a different study. The research firm OptimizeGEO ran 40,000 shopping questions through AI assistants across 12 product categories, without naming any brand, and looked at which sources the assistants relied on for their answers.

Seventy two percent of the sources cited were editorial publishers: reviews, guides and news coverage. A brand's own website earned only about one citation in seven. And in more than 15 percent of cases, the assistants described a product inaccurately, mixing up features, prices or details.

In other words, when an AI decides what to recommend, it listens far more to what others say about a product than to what the company says about itself. A polished product page helps. A thoughtful review in a respected publication helps much more.

What It Means For Brands

The first lesson is that press and reviews have become a sales channel in their own right. Every honest article, comparison and expert review about a product is now raw material for the assistants that millions of people will ask for advice. Earning that coverage is no longer just good publicity. It is how a brand gets recommended at all.

The second is to make the facts impossible to get wrong. Clear names, consistent prices, accurate sizes and plain descriptions, repeated the same way everywhere a product appears, give the software less room to invent. When an assistant describes a product incorrectly, it is often because the information it found was scattered or contradictory.

The third is to plan for the mistakes. If more orders arrive through assistants that misread the task, more of them will come straight back. Simple returns, quick refunds and patient service will decide whether a customer blames the software or blames the brand.

Shopping assistants will get better. The models that pass 82 percent of tasks today will pass more next year. But the habits they are forming now, about which sources to trust and which brands to suggest, are being formed today. The companies that show up in good coverage now will be the ones the machines keep recommending tomorrow.