A 2026 study out of Columbia and Peking University ran field experiments on a major e-commerce platform. They deployed generative AI across seven customer facing workflows and measured revenue impact with millions of users in randomized trials.
The pattern from the experiment was more useful than the results themselves.
The biggest win was a pre-sale chatbot. Before AI, customers got an automated message saying nobody was available. AI filled a complete void. Sales went up 16%. The change was straight forward. More people bought items and cart sizes didn't change. AI answered questions that previously went unanswered, and that was enough.
The worst performer was Google ad title optimization. The existing human titles were already decent. The AI model wasn't tuned for advertising. Google's algorithm ranked the AI titles lower and sales dropped.
The determining factor across all seven was the size of the capability gap AI was filling. Where the gap was wide, AI produced real revenue. Where the existing process was already competent and the model wasn't tuned for the domain, it produced nothing or went negative.
Most teams I talk to start with "where can we use AI?" The research suggests the better question is "where are we currently failing that AI could fix?"
Those are different lists.
Written by Duane Grey
AI Strategy & Implementation
Independent AI consultant helping companies cut through hype and deploy systems that produce real results.