Amazon employees have been sounding off about Q’s struggles with accuracy and privacy since the chatbot’s release this week.
SOPA Images
Amazon Web Services unveiled Q, its business-focused generative AI chatbot, last week.Amazon employees said the chatbot is leaking confidential data, according to Platformer.Q was designed to be a more reliable alternative to consumer-focused AI chatbots.
Amazon’s Q, the AI chatbot for workers its cloud division unveiled on Tuesday, appears to have a few issues.
Q is blurting out confidential information — including the location of AWS data centers, internal discount programs, and even unreleased features — through its internal message channels, like Slack, and ticketing systems, according to leaked internal communications shared with Platformer, a tech newsletter.
The bot is also “experiencing severe hallucinations,” a phenomenon in which AI confidently spits out inaccuracies like they’re facts.
In Q’s case, it’s also been doling out bad legal advice that could “potentially induce cardiac incidents in Legal,” as one employee put it in a company Slack channel, according to Platformer. The bot has also been returning harmful or inappropriate responses that could put customer accounts at risk.
Q’s hallucinations are so “broad and egregious,” as a manager at AWS described them, that the manager even cautioned employees from discussing them in public Slack channels, according to Platformer.
Amazon did not respond to a request for comment from Business Insider but told Platformer in a statement that “Amazon Q has not leaked confidential information.”
It’s not uncommon for generative AI chatbots to falter.
It wasn’t long after Microsoft released its consumer-focused generative AI assistant, Sydney, that it went viral with its own hallucinations. But Q’s transgressions are all the more ironic given that the bot was designed to be a safer and more secure option that businesses could rely on.
Q was built to help workers generate emails, summarize reports, troubleshoot, research, and code. It was also designed to provide them with helpful answers, but only related to content “each user is permitted to access,” Amazon said in a blog post about Q.