A friend of mine recently got hold of an invite-only app called Instinct. It sits inside an ordinary messaging app on his phone, and you talk to it as you would to a colleague. Curious, he began testing it with questions.

The answers unsettled him. It gave him a dizzying amount of personal detail, including his in-laws’ residential address and his national ID and tax ID numbers. As a final test, he asked it to move money from one of his bank accounts to another. It replied that it would need his authorisation, and helpfully supplied a link in the chat where he could give it.

He did not click. He has been thinking about that link ever since, and so have I.

A good design that looks like a bad one: From what my friend and I have been able to establish, Instinct is a legitimate, invite-only product, not a scam. That is what makes the episode worth telling.

In its best light, this is a well-designed assistant. It is capable, but it stops at the moment of consequence and asks the human to authorise the transfer. That is what most of us say we want from artificial intelligence (AI).

Now consider that moment from the user’s side. We have spent years teaching people that this exact sequence -- a familiar sender, personal details and a link........

© hindustantimes