This blog post details configuring smoke tests for Foundry Hosted Agents as part of a CI/CD pipeline. Smoke tests validate basic agent functionality by checking if it generates a response and if that response aligns with the prompt. This differs from unit tests and functional tests, providing a quick way to detect deployment failures before more time-consuming evaluations. The post outlines several essential smoke test scenarios, including verifying on-topic responses, maintaining thread continuity in both stateless and stateful conversations, refusing off-topic questions, resisting hallucinations, and handling context-dependent queries. These tests are defined in a JSON file with prompts and assertion criteria, making them repeatable and easily updatable. The JSON structure allows for multiple tests with varied prompts and assertions. Specific examples demonstrate testing on-topic responses, response chaining using previous_response_id, and conversation-based threading with use_conversation. The scenarios also cover testing when an agent should refuse out-of-scope queries and when it should reject fabricated premises. Finally, a Python script is presented to execute these defined smoke tests, utilizing authentication from the deployment environment or Azure CLI.