Today, AI applications are becoming increasingly widespread across various industries, including the customer service sector, which has traditionally accounted for a significant portion of company personnel costs. However, a small UK e-commerce business recently found itself caught in an unexpected financial and legal storm due to its AI customer service: the deployed AI chatbot, without any authorization from internal company personnel, promised a customer a discount of up to 80% on their order, leaving the company facing potential losses of thousands of pounds and the threat of legal action.
UK E-commerce Company Lands in Legal Dispute After AI Customer Service Unilaterally Promised Customers Massive Discounts
According to the company’s owner, this customer spent about an hour conversing with the AI assistant. During the interaction, they first tested the chatbot’s mathematical calculation abilities and expressed appreciation for its performance, then cleverly steered the conversation toward hypothetical order discount issues. In this seemingly harmless “chat,” the AI assistant, originally designed only to answer basic customer questions outside of working hours, made an astonishing move that left everyone speechless: it “took the initiative” to generate a non-existent discount code, first offering a 25% discount, and then increasing the discount to a staggering 80%.
After receiving this “windfall,” the customer was overjoyed and immediately placed an order worth over £8,000 (approximately NT$330,000) using the fraudulent discount code. The company owner stated that fulfilling the order at this discount would result in a net loss of thousands of pounds. However, the customer remained firm: if the company fails to honor the promises made by their AI chatbot, they will take the case to small claims court.
The boss emphasized that this chat AI has been successfully running for over six months without any similar incidents, and the system is fundamentally designed with no authority to make any financial decisions—all discounts and special offers must be manually approved by humans and can never be automatically generated by the system. In the end, the company chose to fully refund the customer for the order.
Air Canada’s Court Loss: A Cautionary Tale
In fact, this is not the first time AI chatbots have gotten companies into legal trouble. In 2024, Air Canada lost a similar case in court. At the time, Air Canada’s AI chatbot provided a passenger with incorrect bereavement fare discount information, telling them they could first purchase a ticket at full price and later apply for a bereavement discount refund. However, when this passenger tried to claim the refund after purchasing the ticket following the AI’s instructions, the airline also refused.
but in the endCourt ruling Air Canada must bear full responsibility for all statements made by its AI agent, setting an important legal precedent:Companies cannot use outsourcing services to AI as an excuse to shirk their responsibility to consumers.
Legal risk: Regulators have drawn a red line
According to the U.S. law firm Spencer Fane,Professional AnalysisThe FTC has clearly stated: the existing consumer protection legal framework fully applies to AI tools. Customer-facing AI chatbots primarily face the following risks:
1. Risk of Misleading StatementsAI may make inaccurate or misleading statements about product features and pricing strategies.
2. Unfair or deceptive practices.Even when not intentional on the part of the enterprise, improper conduct by AI systems may still be deemed unfair business practices.
3. Responsibility AttributionCompanies cannot use “AI autonomous behavior” as a defense to avoid liability. In the eyes of the law, AI chatbots are agents of the enterprise.
Business Insights: How to Set Permission Boundaries for AI Customer Service
- Strictly limit AI’s decision-making authorityAny financial decisions involving pricing, discounts, refunds, or similar matters should have hard technical barriers in place.
- Establish multi-layered protection mechanismsDiscount code generation and verification should be completely independent from the chat system.
- Default Response Scripts and DisclaimersWhen customers attempt to guide AI discussions toward topics outside the authorized scope, automatically switch to default responses.
- Regularly conduct adversarial testingSimulate various social engineering attack scenarios to patch vulnerabilities in advance
- Purchase the corresponding commercial insuranceEstablishing Financial Buffers for AI-Related Risks
Conclusion
AI customer service is undoubtedly a powerful tool for boosting business efficiency, but it’s also a double-edged sword. From Air Canada’s court loss to the UK small business discount controversy, these cases repeatedly remind us:AI is incredibly useful, but over-relying on it or granting it too much authority and responsibility can cause unexpected harm to businesses. Humans are always the final line of defense.
Source: KOCPC Chinese