AI-text generation tools based on large language models (LLMs) like Google's AI Overview can't actually understand English. Recently I had a very good example of this, so I wanted to share it.
I searched on Google about whether chargebacks are accusations of fraud, because I was writing a very important article you should probably read. Google's AI overview told me "Yes, a chargeback can be an accusation of fraud," and it even had a link with a bunch of sources. However, the AI-generated answer continued to talk about "Chargeback fraud," which, if you speak English, would know doesn't mean the same thing as I asked.
Upon further inspection, I noticed the link that displays the sources for the AI's answer are all about chargeback fraud. None of them actually address the question of whether the chargeback itself is an accusation of fraud, i.e. if I tell my credit card company that I want them to give me the money I sent a merchant, does that mean I'm accusing the merchant of having committed fraud against me? They don't assert such thing verbatim, and yet the AI decided this was going to be the answer.
This is why I CAN NOT TRUST AI.
If you can't "quote" the claim with actual double quotes, you made it up. That thing inside your head where your memories are? That's just a pink sponge of thoughts. It's a miracle it can remember anything at all. I misremember things all the time. I wouldn't trust a person if they can't quote someone, I'm never going to trust an AI that should be perfectly able to cite its claims yet fails so horribly to do so.
A common problem is people's reading comprehension. You may think someone says something because your interpretation of what was written leads you to believe so. If you tell others your own interpretation, you may be passing around your personal distorted view of the facts. Ever heard of the telephone game? People should be able to read the source with their own eyes to make their own judgements. You could be completely wrong about what you believe to be true, and hearing about other people's perspective on it could amend that.
Is this case, the AI's non-existent reading comprehension can't tell apart "a chargeback that is accused of fraud" from "a chargeback that is an accusation of fraud."
Even though none of the links back its claim, it has linked to them in a way so shameless I wouldn't even expect a person to do it. It's confidently incorrect. And if it had a face, I bet it would look pretty smug about it.
By the way, although I found no authority that could assert it in a meaningful way, according to Mastercard, that's one big reason why chargebacks are done in first place, although not exactly the way I meant it:
Here are three common reasons customers dispute card transactions:
https://b2b.mastercard.com/news-and-insights/blog/what-is-a-chargeback/ (accessed 2024-12-07)
- Fraud—Someone unauthorised to use the card made the disputed charge.
- [...]
See how hard that was? All I needed to do was to copy and paste.