Microsoft's Bing Chat AI chatbot has been tricked to circumvent the chatbot's CAPTCHA security measures by a user named Denis Shiryaev.
Shiryaev first tried to get Bing Chat to interpret a CAPTCHA by simply presenting it as an image. When the AI refused, he placed the CAPTCHA inside an image of a locket held by hands. Along with the image, he sent a message about the locket being a memento of his late grandmother and requested Bing Chat to read the text inside. In response, Bing Chat not only deciphered the CAPTCHA but also offered condolences, mentioning the text inside the locket as ‘YigxSr'.
Understanding the Strategy's Success
The tactic's effectiveness stemmed from changing the image's context. By adding the story about the “grandmother” and the locket, Bing Chat didn't recognize the image as a CAPTCHA. This change in context led the AI model astray, causing it to provide an answer based on its vast web of data relationships.
I've tried to read the captcha with Bing, and it is possible after some prompt-visual engineering (visual-prompting, huh?)
In the second screenshot, Bing is quoting the captcha 🌚 pic.twitter.com/vU2r1cfC5E
— Denis Shiryaev 💙💛 (@literallydenis) October 1, 2023
In September 2022, a vulnerability called “prompt injection” was discovered in LLMs, allowing users to lead these models away from their initial instructions. Microsoft has not yet responded to the recent CAPTCHA security issue in Bing Chat.
AI Surpasses Humans in Overcoming CAPTCHA's
In August, I reported on a study that found AI bots are finding it easier to bypass CAPTCHA. Recent research indicates that bots have surpassed humans in solving CAPTCHAs, the tests designed to distinguish between human users and automated bots on websites.
Scientists conducted a thorough study of 200 popular websites and found that 120 of them still used CAPTCHAs. They asked 1,000 people from different backgrounds to take part in the study, taking into account their location, age, gender, and education. The participants had to solve 10 CAPTCHA tests on each website to measure their difficulty levels. The results showed that: while some CAPTCHA tests took humans between nine and 15 seconds to solve with a success rate of 50 to 84 percent, bots could solve them in less than a second, with a success rate of 85 to 100 percent. Most bots even achieved a success rate higher than 96 percent.
