Bob Belderbos: End-to-End Testing Every Rust Exercise with Playwright
The Rust platform has numerous exercises, each requiring a user to load an editor, write code, and validate it via a Rust backend. This process necessitates end-to-end testing to ensure updates don't break existing functionality. Manual testing is inefficient and doesn't cover the full user loop from login to receiving feedback. While unit tests exist for the Django app and Rust validator, they don't test the integrated experience of an exercise. Playwright, combined with pytest, offers a concise solution for this end-to-end testing. A single Python test function is parameterized by fetching all public exercises and their solutions from the database. Each parameterized test navigates to an exercise, injects the correct solution into the editor, and validates the submission. The test asserts that the feedback received indicates a successful completion.Session-scoped fixtures in pytest are utilized to optimize performance by launching the browser and logging in only once for all tests. A crucial aspect of Playwright testing involves handling element timing, particularly for the CodeMirror editor, by using explicit waits for the element's visibility and its JavaScript instance to initialize. A workaround was implemented to avoid Django's asynchronous context trap when creating test users by using a separate fixture that runs before the Playwright fixtures, leveraging django_db_blocker. The end-to-end tests are designed to run against a live database and the actual Rust validator, ensuring realistic integration testing. These tests are executed locally before major pushes, while CI environments primarily run faster unit tests. This approach automatically tests newly added exercises without requiring new test code, and any frontend modification is validated against this integrated regression suite. Playwright is preferred over Selenium for its modern and ergonomic interface, with element timing being a manageable challenge.
django_db_blocker. The end-to-end tests are designed to run against a live database and the actual Rust validator, ensuring realistic integration testing. These tests are executed locally before major pushes, while CI environments primarily run faster unit tests. This approach automatically tests newly added exercises without requiring new test code, and any frontend modification is validated against this integrated regression suite. Playwright is preferred over Selenium for its modern and ergonomic interface, with element timing being a manageable challenge.