We’re gorging on borrowed trus... Note

We’re gorging on borrowed trust and it’s going to cost us.

The text explores the dangerous implications of AI's confident pronouncements, especially when coupled with established brand trust. When designing an AI product's onboarding, the urge to employ deceptive tactics like fake loading animations to impress users was debated but ultimately rejected due to concerns about misleading users. This highlights a core issue: humans often trust confidence over accuracy, a tendency exploited by AI that delivers incorrect information with the same assuredness as correct answers. Unlike humans, AI models can become more confident after making mistakes, lacking the natural hesitation cues we've learned to rely on. Deceptive patterns, like hidden unsubscribe buttons, manipulate behavior, but making users believe the AI is trustworthy before it earns it is a far more insidious form of deception. Historically, confidently wrong individuals faced consequences, making trust in confidence a reasonable strategy. However, AI's scale and unreadability amplify this risk. Authority bias leads people to trust information based on its source, even against their own judgment, as demonstrated by historical experiments. Productized experts, stripped of their human hesitation, offer conclusions at scale, but this removes a crucial element of trustworthiness. Companies leveraging trusted names for AI risk squandering decades of earned credibility with a single, confidently wrong output. Google's expensive stock drop after a Bard error and Air Canada's legal liability for its chatbot's fabricated policy illustrate the severe financial and reputational costs. The asymmetry of trust means it builds slowly but can collapse rapidly, with AI exacerbating this by making errors at an unprecedented scale under a trusted brand's banner.
CdXz5zHNQW_Lip4n5VBXa.jpeg