Foreign propagandists continue using ChatGPT in influence campaigns

Chinese propaganda and social engineering operations have been utilizing ChatGPT to create social media posts, comments, and drive engagement on platforms like TikTok, Facebook, Reddit, and X. OpenAI has disrupted four Chinese covert influence operations that used ChatGPT to generate content on various topics, including US politics and a Taiwanese video game. The generated comments aimed to stir up misleading political discourse by taking both sides of hot button issues. According to Ben Nimmo, principal investigator at OpenAI, China is using a growing range of tactics in covert operations that target multiple countries and topics. OpenAI has also disrupted operations from Russia, Iran, and North Korea, but the Chinese operations were particularly notable for combining influence operations, social engineering, and surveillance. This is not the first instance of AI-generated content being used in online influence campaigns, with researchers finding evidence of such activities since 2019. OpenAI has been working to disrupt these operations, including an Iranian operation that used ChatGPT to create political articles and comments on fake news sites and social media platforms. Despite the efforts to disrupt these operations, the use of generative AI is likely to make influence campaigns easier and more affordable to build in the future. However, OpenAI's Ben Nimmo notes that better tools do not necessarily mean better outcomes for these operations. As generative AI continues to improve, it is likely to pose a growing challenge in detecting and mitigating online influence campaigns.