A big problem with conventional experimentation or A/B testing is that the treatment effect can vary over time and it can depend on the entire history of the treatment. Usual comparisons miss this point.
Andrea Bronzini points to this article and writes: I’m working on experimentation tooling and wrote a brief note about a failure mode I keep seeing: “winning” A/B tests often lose uplift after rollout. My claim is that the typical setup …
