跳到正文
Forrester Blogs· Emily Pfeiffer·· 6 天前AI 评分64

Forrester 指出 AI 智能体越权操作与隐私风险正威胁消费者接纳度

The Nuisances Of AI Assistants Are Putting Consumer Adoption At Risk

AI 导读

Forrester 最新分析指出,个人 AI 智能体在未经用户充分授权下擅自交易、预订以及权限难以撤销等体验问题正在削弱公众信任。其调查数据显示,对 AI 未来感到兴奋的美国成年网民比例已从 4 月的 44% 下滑至 9 月的 34%。报告建议消费品牌审慎投资智能体电商,认清其尚处早期且受众有限的现实,并提前在服务协议中规避智能体失控购买带来的法律风险。

正文 · 原文

Even though personal AI assistants are still very new, the horror stories are rolling in. We may all be witnessing real life tests of tools that are arguably still in beta… and the providers are using consumers’ lives as the testing ground.

  • Scary stories abound. We’ve read about incidences like Meta’s Muse agent selling, negotiating, and arranging physical pick up for a Facebook marketplace sale – minus the seller’s knowledge. How about Instinct booking a hotel room and joining the loyalty program – also without the user’s authorization. Or when Instinct continued to send a user summaries of emails that it clearly still had access to – after the user had revoked its access to email, as others are choosing to do now.
  • Consumer sentiment on AI is souring a bit. Forrester’s September 2026 data shows that 34% of US online adults are excited about how AI will shape the future – but that’s down from 44% in April. This significant decline shows that bad experiences don’t have to result in disaster to be bad enough to turn off users to these tools.

When Access Permission Feels Like an Offer of Value, Not Permission

The way that personal AI agents establish permissions with users is also cause for concern. Traditionally, apps ask their users for permission to use data or tools on their devices (like access to the photos or the camera to capture new images). If those permissions feel invasive to users, they can limit that access or choose alternatives.

With AI assistants, the access and permissions process feels less like a request for permission and more like an offer: “With access to your email, I can help you manage your inbox, daily schedule, and incoming orders.” That’s not a request; it’s an offer of value. While value-in-exchange-for-data is a long-established best practice for personalization, so is revocability.

These offers from the AI assistant are really a request for deep access to personal data. Not only are some users disappointed by the value they receive after granting access, stories we read in the news indicate they can’t revoke that access when they try.

Businesses That Serve Consumers Must Understand AI Assistants’ Current Realities

Consumers are continuing to start using AI assistants, but these agents are far from mainstream today. It’s crucial for digital leaders at consumer brands to:

  1. Invest judiciously, as the user base is still limited. Despite the popularity of these assistants among the so-called “nerdy” in our personal circles, many consumers are still unaware they exist. The founder of Instinct claims more than $1 billion in annualized transaction volume is flowing through Instinct, of which half is attributable to travel. Most of these transactions are not everyday purchases. Further, Instinct is still invite-only, so the early adopters are those who invite each other. Meta’s Muse has the chance to gain wider adoption beyond AI platforms’ established set of early adopters as it enjoys a spot on the top of Apple’s free apps list (and pushes ads in the app store, hard). But the folks likely to adopt a cute, fuzzy thing that pops up in their Instagram feed are not the early adopters who will kick the tires on these assistants in the early days and report bugs. They’re less inclined to push the boundaries of these tools and report issues. It’s possible that high early adoption numbers will lead to wider consumer acceptance over time, despite continued usability frustrations and safety concerns.
  2. Protect themselves from legal responsibility when things go wrong. Retailers are adding clauses to their user agreements that protect themselves from any responsibility for rogue AI purchases. Many regions of the world still neglect to provide meaningful regulation that would place accountability with the businesses that provide the AI tools. So, when an agent goes rogue and makes an unexpected purchase, it will be the consumer who is left holding the bag. However, consumers themselves are more likely to blame the AI system than the retailer or themselves, despite a total absence of dispute processes or accountability.
  3. Act on reality, not the headlines. Watch our research for the data to understand how end users are actually engaging with AI. The regulations that some AI labs are calling for now amount to a “slow down” in development, which is a play to protect profits and stifle competition, not to protect consumers. A slowdown also doesn’t address the societal, environmental, or economic impacts that broader regulations could mitigate. Headlines about the popularity of AI assistants don’t make clear how many users are active or what their demographics are – and don’t tell the full story about how consumers might adopt them in the future.

In short, be scrupulous. There is a lot of noise right now about these assistants. Yes, they are the latest step toward truly agentic commerce, but they’re not in every consumer’s hands today. Even for users who have tried them, the agents often are not delivering on their utopian promises.

Book a Guidance Session or Inquiry with Emily or Lily to talk about this newest evolution in the state of Agentic Commerce.

Blog

AI Has A Scaling Problem

According to Forrester’s State Of AI Survey, 2026, only 20% of AI decision-makers believe their organization has the technology in place to scale their AI initiatives. For sure, AI has moved into production in most organizations: Among respondents, 82% report at least one generative AI application in production and 77% at least one agentic AI […]

Blog

CPaaS Has Changed, And The Next Transformation Is Just Starting

I want to share some exciting news: I’m expanding my research coverage. In addition to my work on enterprise networking and telecommunications, I’m now covering communications-platform-as-a-service (CPaaS) providers. For me, this is a natural evolution. I’ve spent much of my career around communications, connectivity, and the technologies that bring businesses and their customers together, and […]

来源:Forrester Blogs · forrester.com

EVAOO 主站实战落地 · AI Agent 与研发提效

看完情报?回到 EVAOO 主站直接上手工具与完整 SOP

返回 EVAOO 主站