I hate to admit it, but I spent a lot of money online this holiday shopping season. And it’s no surprise that some of the purchases don’t meet my expectations. The photo book I bought was damaged in transit, so I took a few photos, emailed them to the seller, and got a refund. Online shopping platforms have long relied on customer-submitted photos to verify that refund requests are legitimate. But generative AI is now starting to break that system.
A Very Suspicious Pinch
On Chinese social media app RedNote, WIRED found at least a dozen posts from ecommerce sellers and customer service representatives complaining about AI-generated refund claims they received. In one case, a customer complained that the bed sheet she had purchased was torn to pieces, but the Chinese characters on the shipping label looked like a curse. In another, a consumer posted a photo of a coffee cup with cracks that look like paper tears. “This is a ceramic mug, not a cardboard mug. Who would tear a ceramic mug into pieces like this?” the merchant wrote.
Retailers reported that there are several product categories where images of AI-generated damage are the most abused: fresh groceries, low-cost beauty products, and fragile items like ceramic mugs. Sellers typically don’t ask customers to return these items before issuing a refund, making them prone to return scams.
In November, a crab seller on Douyin, the Chinese version of TikTok, received a photo from a customer that made it appear that most of the crabs he bought arrived dead, while two others had escaped. The buyer even sent videos showing dead crabs being pushed around with a human finger. But something was off.
“My family has been raising crabs for more than 30 years. We have never seen a dead crab with its legs up,” said Gao Jing, a seller, in a video he later sent to Douyin. But what finally pulled off the con was the gender of the crabs. There were two men and four women in the first video, and the second clip had three men and three women. One of them had nine legs instead of eight.
Gao later reported the fraud to the police, who discovered the videos were indeed fake and detained the buyer for eight days, according to a police notice Gao shared online. The case attracted a lot of attention on Chinese social media, in part because it was the first known AI refund scam of its kind to provoke a regulatory response.
Lowering Barriers
This problem is not only found in China. Forter, a New York-based fraud research firm, estimates that AI doctor images used in reimbursement claims have increased by more than 15 percent since the beginning of the year, and continue to grow globally.
“This trend started in mid-2024, but has accelerated over the past year as imaging tools have become more widely accessible and incredibly easy to use.” said Michael Reitblat, CEO and founder of Forter. He adds that AI doesn’t need to get everything right, as front-line sales staff and return review teams may not have time to scrutinize each image.
