Evaluating LLM outputs in production requires more than simple operational checks like HTTP status codes. The Air Canada chatbot incident highlights how a 200 status code can mask harmful hallucinations, like inventing a discount policy. To address this, a separate evaluation layer is needed to assess output quality independently of operational health. This layer uses LLM-as-a-judge with scored rubrics to evaluate dimensions like correctness and relevance. System prompts for these judges must clearly define their role, output format, and embed detailed, consistent scoring rubrics. These rubrics, written in plain language, provide criteria and rating scales for assessing LLM responses. Techniques like G-Eval can improve reliability by breaking down evaluation into sequential steps. Runtime guardrails act as a safety net, intercepting and flagging or blocking unsafe or irrelevant outputs at inference time. These guardrails often employ LLM-as-a-judge with specific rubrics designed for immediate policy enforcement. Implementing a production evaluation workflow involves versioning system prompts and rubrics, running offline evaluations, and using guardrails as a runtime safety net. This creates a continuous feedback loop for prompt tuning and quality maintenance. By integrating system prompts, scored rubrics, and runtime guardrails, LLMs can be managed effectively in production to ensure both quality and safety.