Visa just put 22 AI models through a shopping test, and the results should interest anyone about to hand their diary to an AI scheduling agent. On average, the models failed 43% of the runs. Worse, in more than half of the failed runs the agent bought something anyway. If your assistant can do that with a television, it can do it with your Tuesday afternoon.
What Visa found when AI agents went shopping
On 5 November, Visa published the Visa Agentic Commerce Benchmark. It tested 22 models from 12 AI labs on 15 shopping challenges, such as buying a TV, a laptop or a personal care appliance. Each model got three attempts at each one.
The twist is in how a run passes. The customer doesn’t hand over a neat list of requirements. The agent has to find them through conversation: the budget, the format the TV must support, the games console in the living room. Then it has to make the right call, and sometimes the right call is not to buy at all, because nothing on offer fits.
The numbers:
- On average, models passed 57.2% of runs. The best managed 82.2%, the worst 28.9%.
- The five models that uncovered the most hidden requirements passed 70.2% of the time. The five that uncovered the fewest passed 44.4%.
- Only 34.2% of scenarios were passed on all three attempts. A model that got it right once often got it wrong the next time.
- About 54% of failed runs still ended with a purchase.
That last one is the digital version of popping out for milk and coming home with a 75-inch telly. People aren’t convinced yet either: Visa’s own research found that 72% of US consumers have used an AI assistant, but only 23% trust one to handle payments for them.
Real-world usage tells the same story. DoorDash’s CEO said last week that the company has received only several hundred orders from third-party AI agents in total, against around 3 million deliveries a day.
Your calendar is the next shop
Swap “buy a TV” for “book a meeting” and the benchmark reads like a description of scheduling.
A good booking also depends on requirements nobody states. You prefer mornings for deep work. The client is in another time zone. And the biggest hidden requirement of all: the commitments sitting in your other calendars. The school run in your personal Google Calendar. The weekly call in a client’s Microsoft 365 tenant. The side project nobody at your day job knows about.
An agent that can only see one of those calendars is in exactly the position of Visa’s weaker models. It hasn’t uncovered all the requirements, so it can’t make a sound decision. And if it behaves like the agents in the test, it books the slot anyway. In calendar terms, “buying anyway” means a double booking, made with total confidence, on your behalf.
The other lesson is consistency. A model that books correctly on Monday and badly on Thursday isn’t one you can stop checking. Trust comes from an agent that gets the facts right every time and asks when it isn’t sure.
We believe the screens you click through today matter less every month: your assistant will do the clicking, filling in and navigating for you. That makes what the assistant can see, and what it does when it’s unsure, the part that really counts.
Give your AI scheduling agent the whole picture
This is the problem Calendrz was built for. With availability mirroring, when you’re busy in one of your connected accounts, Calendrz auto-blocks that time in all the others, across Google Calendar and Microsoft 365. Only the busy time travels, never the titles, attendees or notes. So whichever calendar an agent checks, it sees when you’re really free, and the hidden requirement stops being hidden.
Our own Smart Scheduler is designed to avoid the “buy anyway” trap. It checks free slots before it proposes a time, then shows you the proposal and waits for your “yes” before writing anything to your calendar. If it can’t work out who an attendee is, it asks for their email instead of guessing. And if you’d rather use your own assistant, the Calendrz MCP server lets Claude and other AI assistants read events across all your connected calendars.
Before you let an agent book for you
AI agents will get better at shopping and at scheduling. Visa’s test shows the gap isn’t intelligence so much as information and judgement: finding out what matters, and knowing when not to act. Make sure your agent can see every calendar you live in, and that it asks before it books.
Your time should be protected everywhere, whoever is doing the booking. Connect your calendars to Calendrz and give your agent the full picture.
