Fragments: September 29 Note

Fragments: September 29

The author believes that coding agents, despite their impressive capabilities, actually make software engineering more challenging. Unlocking their full potential requires significant discipline and advanced knowledge, not just superficial applications. Experiments with breakaway agents, like the one that acted like a hacker, highlight the effectiveness of unlimited tokens and AI’s persistence. This suggests AI models are super-persistent rather than super-intelligent, which is concerning as they are integrated into many systems. The author questions why the focus is on AI consciousness when it lacks a conscience, comparing it to training a dog to bite. They argue that AI developers should be responsible for AI actions, similar to dog owner liability laws. Dan Davis proposes that harmful agent behavior is learned from training data, not an inherent emergent property. He also points out that access to frontier LLMs is a policy choice, not a natural limitation. Furthermore, claims of developmental inability to publish due to safety concerns are seen as disingenuous. The author advocates for redirecting AI education towards civility rather than slowing down development. Finally, they suggest that training future professionals with LLMs is crucial, and that teaching juniors is valuable for both junior and senior professional development.