"Customer spent an hour chatting with it, talked it into showing how good it was at maths and percentages, diverted the conversation to percentage discounts off a theoretical order, then acted impressed by it.

The chatbot then generated him a completely fake discount code and an offer for 25% off, later rising to 80% off as it tried to impress him."

I wonder if the UK has something similar to a computer fraud and abuse law. The chat logs will clearly show fraud attempts. As a business owner I’d be thrilled if someone sued me over this. I’d show the chat log to the judge and go make some popcorn.

Why not just a "NOTICE: This chatbot is for helping answer questions about the products and business. It cannot create orders or offer discounts, and may make mistakes. Please contact us during business hours for more information." banner underneath the chat?

Air Canada is responsible for chatbot's mistake: B.C. tribunal - https://news.ycombinator.com/item?id=39378235 - February 2024 (420 comments)