Inside the AI Prompts DOGE Used to “Munch” Contracts Related to Veterans’ Health

The Department of Government Efficiency (DOGE) used an AI script to review contracts for the Department of Veterans Affairs (VA) as part of President Trump's executive order to cancel unnecessary contracts. The AI script, written by Sahil Lavingia, was given unclear and conflicting instructions, leading to mistakes. The script was supposed to cancel contracts that weren't "directly supporting patient care," but neither Lavingia nor the model had the knowledge to make such determinations. Experts found numerous flaws in the code, including the use of older models not suited for the task, and the model's tendency to "hallucinate" contract amounts. The experts also criticized the use of off-the-shelf AI models with little context on how the VA works. Lavingia acknowledged the flaws and blamed a lack of time and proper tools. The AI model was given instructions that were unclear and sometimes contradictory, leading to mistakes in flagging contracts for cancellation. Experts said that the model's mistakes were due to a lack of understanding of how the VA works and what constitutes "direct patient care." The mistakes led to contracts for important services like internet connectivity being flagged for cancellation. Overall, the experts concluded that the use of AI in this way was prone to errors and should not have been attempted without more careful planning and expertise.