Glyph Lefkowitz: Adversarial C... Note

Glyph Lefkowitz: Adversarial Communication

AI systems make unpredictable mistakes, necessitating the verification of every output. This verification process can be as costly as the original task, prompting the externalization of costs onto others. A key concern is the "reverse centaur" phenomenon, where employers turn employees into unpaid AI verifiers. This occurs because AI output errors are a generalized externality, making it cheaper for humans to do the work entirely. The claim that "code review is the bottleneck" highlights the need for human understanding, not just technical review. In a competitive environment, AI aids "ladder-climbers" by allowing them to produce large amounts of code quickly while offloading the verification burden onto others. This dynamic benefits the prompter and risks major issues, with the reviewer often blamed. AI also excels as a "Gish Galloper," overwhelming opponents with a deluge of falsehoods, particularly in political discourse. Fraud, spam, and scams are facilitated by AI's ability to generate convincing text rapidly, externalizing verification costs to victims. Customer support bots, often seen as adversarial, use AI to quickly dismiss or stall customers, prioritizing metrics over resolution. In education, AI enables widespread cheating, as students externalize the verification work onto teachers. While some imagine AI being used for legitimate appeals against powerful entities, such as insurance companies, the structural advantages of these entities often prevail. Even beneficial uses of AI can escalate into an arms race, potentially leading to worse long-term outcomes. AI's summarization and question-answering capabilities can also be weaponized, though the act of reading itself is seen as more destructive to website content.