Many AI experts privately express concern about AI's potential to enable the creation of a silently spreading, deadly pathogen. This worry stems from the belief that advanced AI models could assist malicious actors in developing such a pathogen before robust detection and prevention systems are in place. While not a certainty, it is considered a plausible scenario as AI improves its understanding of biological vulnerabilities. A recent MIT and University of Queensland study of 272 researchers found a 12% chance of AI's dangerous capabilities leading to a catastrophic outcome by 2030, which includes the construction of biological weapons. This risk rises to over 20% without mitigation efforts. Leaders from major AI companies have publicly warned about AI-derived bioweapons and called for safeguards. The federal government and industry are debating how to review new AI models to prevent misuse, hoping developers can build guardrails to block bad actors. However, preventing malicious use is challenging, especially with rapidly improving open-source models that can be adapted outside regulatory oversight. The feared scenario involves AI mapping human biological vulnerabilities, identifying genetic tweaks to existing pathogens, and generating viable candidates. These could then be synthesized using accessible gene-editing tools, leading to a novel pathogen that spreads undetected until containment becomes extremely difficult. This highlights the urgent need to develop effective detection and prevention methods faster than new offensive capabilities emerge, and underscores the importance of the open-source debate regarding AI safety.