ML models degrade over time without evaluation, a problem often overlooked for conversational data agents. Genie Ontology, specifically through Genie Agent Benchmarks, addresses this by continuously evaluating and improving agents. Benchmarks involve isolated conversations, each evaluated in one of two modes depending on the question type. Chat mode compares the agent's generated SQL or its result against a "ground truth" answer, with objective rules for "Good" and "Bad" results. Agent mode is for multi-step, text-based reasoning reports, where an LLM judge evaluates the response against an optional evaluation note. A key design detail is Chat mode's ability to reward legitimate variations while flagging structural errors, fostering trust in the benchmark. Best practice recommends registering two to four phrasing variations per business question to ensure comprehensive testing. After a benchmark run, Genie Code analyzes the results, proposing instruction or context adjustments to address identified gaps. This "batch review" also applies to real usage data, enabling proactive problem-solving. However, human review is crucial as user feedback doesn't automatically change agent behavior. The quality of ground-truth SQL is also critical, and the tool doesn't store long-term accuracy trends internally. Investing in benchmarks from day one is essential to efficiently validate changes and ensure the agent's ongoing reliability.