Airbnb's Forecasting Data Science team produces critical company-wide forecasts. These forecasts are influential, so even small biases can have significant consequences. The team has learned to distinguish between three distinct actions when a forecast model struggles: refitting, respecifying, and holding. Refitting involves updating a model with new data while keeping its structure and features the same. This is the cheapest option but carries a risk of degrading performance if recent data is unusual. Respecifying means changing the model itself by adding or removing features or altering its structure. This is a more significant undertaking but is necessary to correct fundamental model errors. Holding, or doing nothing, requires the most confidence as it means leaving a model untouched despite apparent misses. This is often the correct choice when misses are within the model's expected range or when the cause of the miss is temporary. The failure to differentiate these actions can lead to issues like chasing noise, carrying stale assumptions, or overcorrecting during periods of disruption. Deciding which action to take now hinges on understanding what has fundamentally changed about the underlying data generation process. If the process remains the same and only parameters have drifted, refit. If the process has changed in a way the model cannot capture, respecify. If the miss is within normal bounds, hold.