Cloud CISO Perspectives: Recent advances in how threat actors use AI tools

Google Threat Intelligence reports a rise in threat actors misusing AI, moving beyond productivity to active malware deployment. New malware, PROMPTSTEAL, uses LLMs to generate commands for data theft, marking the first observed LLM query in live operations. Threat actors are employing social engineering tactics to bypass AI safeguards like Gemini, posing as researchers to elicit sensitive information. An illicit AI tool marketplace is emerging, offering phishing, malware, and vulnerability research tools, lowering barriers for less sophisticated actors. State-sponsored groups from North Korea, Iran, and China are using AI for reconnaissance, phishing, C2 development, and data exfiltration. Google is countering these threats by disabling malicious accounts and improving its AI models to resist misuse. The company emphasizes its commitment to responsible AI development through security measures and continuous model testing. Recent updates include threat modeling practices, quantum-safe computing preparations, HTTPS by default for Chrome, and AI-powered Android scam protection. Lastly, privileged account monitoring and Pro-Russia information operations are highlighted as areas of concern.