The author initially developed a GitHub App, LogoMesh, designed to perform adversarial testing on Python code changes within pull requests. The app aimed to identify bugs by generating adversarial inputs and running tests in a hardened Docker sandbox. Despite the app's advanced features and technical sophistication, it encountered significant problems. The app's silence rate on real pull requests was high, and many reported findings were irrelevant artifacts. Crucially, the app's performance was slow, with high latency, exceeding the desired timeframe. This led the author to realize a fundamental flaw: building without validating the initial assumptions. The author shifted their focus after realizing the true pain point wasn't preventing future bugs but rather dealing with existing production bugs. This revelation led to a simpler solution: a tool that generates a failing pytest based on a Sentry URL, reproducing the crash. This tool leverages the core technology developed for LogoMesh, like the Docker sandbox, but with a different purpose. Currently, the author is evaluating whether creating a reproducing test before fixing a production bug is a universal problem. They seek validation to ensure they are solving a real pain point instead of building another tool without proper user input. The author questions if they are solving a real problem or merely addressing a personal workflow inefficiency.
dev.to
dev.to
Create attached notes ...
