Testing is a critical part of delivering quality software to your customers. Testing frameworks allow testers to write code that tests the app before it’s published. The framework acts like a user by pressing buttons, entering text and checking the results. Testing frameworks can make automated testing really easy or really annoying. Choosing the right framework makes a big difference in how well your team will test your app. So choose wisely!
Table of Contents
Primer on Playwright
Playwright was created by Microsoft in January 2020 as a fully featured, multi-browser testing framework. It supports most programming languages and offers features like tracing, replay, loading-detection and code generation. Playwright aims to provide a level of polish and professionalism to an ecosystem that has a lot of open source and low-level solutions. It really is a great choice for testing teams.
Playwright is a fully featured framework with everything from multi-browser support, locator utilities and trace tooling. This really is the whole package. There isn’t enough time to get into every little feature, but here’s a high level overview. It will serve your team well.
Cross Browser and Cross Platform Testing
You can write a single test script and run it on Chrome, Firefox, Safari, Edge or even on mobile! This is a huge win because you don’t have to worry about browser compatibility. Write your tests once and test everywhere. But wait, there’s more.
Run your tests on Windows, Mac or Linux without any special setup. That means you can run this in CI, on your development machine or on your friend’s development machine. This is all available out of the box! Amazing.
Element Loading Detection
In the background, Playwright has some great anti-flakiness features like load detection. When your test is looking for an element, it may not be loaded yet causing most tests to error out. Playwright, however, will automatically wait until everything is loaded before attempting to make your assertion. This reduces the amount of flakes and gets rid of all those hardcoded wait statements you were using before.
Tracing and Codegen Tooling
Playwright comes with some great tooling out of the box. Their tracing tool allows you to view the test history and inspect the DOM, console logs and network requests. The tracing tool comes packaged in a polished and easy to use UI. This tracer will make your debugging way easier and allow you to share better details with your developers.
The codegen tool is really useful too. Codegen records your manual test session and writes the code to reproduce that. This is a great way to build tests quickly and efficiently.
Pros of Playwright
Multi-browser and Multi-Platform support
The cross browser and cross platform is a huge win. There are no more restrictions on what you can test and where. Write your tests once and run them anywhere.
Comprehensive Automation Capabilities
Playwright has all the features you would expect from a test framework. You can get the job easily with their anti-flakiness, locator utilities and comprehensive API. No more hacking around the framework to get things to work correctly.
Robust Documentation and Active Community
Cons of Playwright
Longer Learning Curve (especially for beginners)
The downside of being an extensive framework is that it takes a bit to learn. Beginners may find the API overwhelming and difficult to get started with. The documentation is good, but there is just so much to learn. Hopefully courses and tutorials will help streamline this over time.
Limited Ecosystem (compared to other frameworks)
Primer on Cypress
Cypress was made public in October 2016 as an improved testing framework (compared to the tools at the time). Cypress is a fully featured automation framework with anti-flakiness, locator utilities and debug tooling. This framework has grown and improved over the years and has a fairly large 3rd party ecosystem.
This framework is fully featured and very capable. Cypress comes with a full list of standard automation functions and extensive 3rd party integrations that fill the remaining gaps. You and your team would be well served with this choice.
Debugging and Time Travel
Writing tests for Cypress is really straightforward. The API is very readable and simple – there are like 4 main function calls for most tests. You can dive deeper with stubs and backend access, but the main API is very simple and straightforward.
3rd Party Integrations
Pros of Cypress
Easy to Learn and Get Started
Powerful Debugging and Time-Travel
Cons of Cypress
There are some downsides to using Cypress as well. It’s not the best choice for everyone. Some of these may be deal breakers for you.
Limited to Chromium-based Browsers
Doesn’t Support Parallel Test Execution
Limited First-Party Support
Cypress doesn’t have great first party support for a lot of things (hence the plugin ecosystem). This lack of first-party support means that you end up using a lot of plugins that each have their own flavors, trade offs and bugs. There isn’t a unified API that deals with all the different aspects of testing, it’s really a collection of solutions that are cobbled together.
In terms of learning curve, Playwright is more complex to learn. Cypress is much easier to get started with. Cypress may be a better choice for teams with a lot of turn over, or low technical capability. Playwright may be a better choice if your team is able to climb the learning curve and get the rewards of a unified API.
Debug and Codegen Tooling
Playwright has great debug and codegen tooling, Cypress uses the amazing Chrome Devtools and has a beta-level codegen tool. Chrome Devtools are familiar and really useful. But, the Playwright development tool is “good enough” in my opinion. I wouldn’t choose Cypress over the Chrome Devtools. The codegen tool on Playwright feels a lot more polished and capable than the Cypress one.
Ecosystem and Community
Parallel Test Execution
Where Playwright is a Better Choice
Playwright is the general better choice. You have a longer learning curve. The advantages of full browser support, better tooling and unified API make it worth it in my opinion. Most companies aim to retain testers for a number of years, so they can make the investment on that initial learning curve.
Where Cypress is a Better Choice
Cypress may be a better choice in a few specific situations. Companies that have high turnover or low technical ability in their testing department could really benefit from Cypress. The quick start time makes it easy to get people up to speed and contributing. Companies who don’t care about cross-browser support or longer test times may also find Cypress more attractive. Cypress isn’t a bad choice, I just think Playwright is better.
Overall Cypress and Playwright both have a lot of offer. A team will be well served by either. In general, I would recommend using Playwright. Playwright has better capabilities, more resources (from Microsoft) and is a more polished product. Cypress is good, but lacks a cohesive structure and fully fleshed out API.