Testing is a critical part of delivering quality software to your customers. Testing frameworks allow testers to write code that tests the app before it’s published. The framework acts like a user by pressing buttons, entering text and checking the results. Testing frameworks can make automated testing really easy or really annoying. Choosing the right framework makes a big difference in how well your team will test your app. So choose wisely!
Table of Contents
Playwright and Cypress frameworks are both popular choices. They both are fully featured and have substantial community support. They’re pretty comparable when we compare the star history on Github. So what should you pick? How do you decide? Let’s dive into the pros and cons of each.
Primer on Playwright
Playwright was created by Microsoft in January 2020 as a fully featured, multi-browser testing framework. It supports most programming languages and offers features like tracing, replay, loading-detection and code generation. Playwright aims to provide a level of polish and professionalism to an ecosystem that has a lot of open source and low-level solutions. It really is a great choice for testing teams.
Playwright Features
Playwright is a fully featured framework with everything from multi-browser support, locator utilities and trace tooling. This really is the whole package. There isn’t enough time to get into every little feature, but here’s a high level overview. It will serve your team well.
Cross Browser and Cross Platform Testing
You can write a single test script and run it on Chrome, Firefox, Safari, Edge or even on mobile! This is a huge win because you don’t have to worry about browser compatibility. Write your tests once and test everywhere. But wait, there’s more.
Run your tests on Windows, Mac or Linux without any special setup. That means you can run this in CI, on your development machine or on your friend’s development machine. This is all available out of the box! Amazing.
Element Loading Detection
In the background, Playwright has some great anti-flakiness features like load detection. When your test is looking for an element, it may not be loaded yet causing most tests to error out. Playwright, however, will automatically wait until everything is loaded before attempting to make your assertion. This reduces the amount of flakes and gets rid of all those hardcoded wait statements you were using before.
Tracing and Codegen Tooling
Playwright comes with some great tooling out of the box. Their tracing tool allows you to view the test history and inspect the DOM, console logs and network requests. The tracing tool comes packaged in a polished and easy to use UI. This tracer will make your debugging way easier and allow you to share better details with your developers.
The codegen tool is really useful too. Codegen records your manual test session and writes the code to reproduce that. This is a great way to build tests quickly and efficiently.
Community Support
Microsoft has put a lot of resources into Playwright’s development and support. Their repository is actively developed and bugs are addressed reasonably quick. It’s great to know that a framework has good support and will continue to be there in the future.
Pros of Playwright
Multi-browser and Multi-Platform support
The cross browser and cross platform is a huge win. There are no more restrictions on what you can test and where. Write your tests once and run them anywhere.
Comprehensive Automation Capabilities
Playwright has all the features you would expect from a test framework. You can get the job easily with their anti-flakiness, locator utilities and comprehensive API. No more hacking around the framework to get things to work correctly.
Robust Documentation and Active Community
The Playwright site and code packages have full documentation which makes it easy to figure things out. There is an active community of testers using Playwright so you can get help if you’re stuck. And, if you somehow run into a bug with the framework, the community fixes it relatively quickly.
Cons of Playwright
Longer Learning Curve (especially for beginners)
The downside of being an extensive framework is that it takes a bit to learn. Beginners may find the API overwhelming and difficult to get started with. The documentation is good, but there is just so much to learn. Hopefully courses and tutorials will help streamline this over time.
Limited Ecosystem (compared to other frameworks)
Since Playwright is so young, the plugin ecosystem hasn’t really caught up yet. Other frameworks, like Selenium, have more extensive third-party tooling for testers. Over time I would expect the ecosystem to grow and add more plugins for Playwright.
Primer on Cypress
Cypress was made public in October 2016 as an improved testing framework (compared to the tools at the time). Cypress is a fully featured automation framework with anti-flakiness, locator utilities and debug tooling. This framework has grown and improved over the years and has a fairly large 3rd party ecosystem.
Cypress Features
This framework is fully featured and very capable. Cypress comes with a full list of standard automation functions and extensive 3rd party integrations that fill the remaining gaps. You and your team would be well served with this choice.
Debugging and Time Travel
Cypress comes with some great debug tooling that will save you a lot of time. After running a test, you can time travel back to a particular point and inspect anything you would like. Since Cypress runs within the browser, you can use the Chrome Developer tools that you’re already familiar with.
Simple API
Writing tests for Cypress is really straightforward. The API is very readable and simple – there are like 4 main function calls for most tests. You can dive deeper with stubs and backend access, but the main API is very simple and straightforward.
3rd Party Integrations
Cypress has over 160 plugins for everything from reporting to test utilities. The plugin community is largely grass-roots driven and is actively developed. You can be sure that if you have a niche use-case, they’ll have something to support you.
Pros of Cypress
Easy to Learn and Get Started
It really is quite easy to get started with Cypress. If you learn ~4 functions, you can be off to the races. Since everything is in javascript, all the documentation and articles online are easy to digest and understand.
Powerful Debugging and Time-Travel
Cypress users use the powerful set of tools available in Chrome Devtool. It really makes debugging easy when you pair the dev tools with Cypress’ time travel feature. The debugging experience is pretty great.
Strong Community
Cons of Cypress
There are some downsides to using Cypress as well. It’s not the best choice for everyone. Some of these may be deal breakers for you.
Limited to Chromium-based Browsers
Doesn’t Support Parallel Test Execution
The way that Cypress was built prevents it from running tests in parallel. This is a really big limitation for large test suites since it makes test runs take a lot longer. Some testers are able to get around this by splitting tests up into separate suites and running them on separate machines.
Limited First-Party Support
Cypress doesn’t have great first party support for a lot of things (hence the plugin ecosystem). This lack of first-party support means that you end up using a lot of plugins that each have their own flavors, trade offs and bugs. There isn’t a unified API that deals with all the different aspects of testing, it’s really a collection of solutions that are cobbled together.
The Throwdown
Browser Support
Let’s talk about multi-browser support. Playwright has support, Cypress doesn’t. This is a big deal. If you’re going to test on more than one browser, you want to test on them all. For many people this is a deal breaker.
Capabilities
Playwright has a far more extensive API, Cypress has far less capabilities out of the box, but can bridge some of the gaps with plugins. It is quite a bit easier to work with a single, extensive API that is well documented. I would prefer Playwright’s approach here.
Learning Curve
In terms of learning curve, Playwright is more complex to learn. Cypress is much easier to get started with. Cypress may be a better choice for teams with a lot of turn over, or low technical capability. Playwright may be a better choice if your team is able to climb the learning curve and get the rewards of a unified API.
Debug and Codegen Tooling
Playwright has great debug and codegen tooling, Cypress uses the amazing Chrome Devtools and has a beta-level codegen tool. Chrome Devtools are familiar and really useful. But, the Playwright development tool is “good enough” in my opinion. I wouldn’t choose Cypress over the Chrome Devtools. The codegen tool on Playwright feels a lot more polished and capable than the Cypress one.
Ecosystem and Community
The ecosystem around Cypress is large and well-established. But, Playwright has a growing ecosystem and community. I think in a few years Playwright will have caught up to Cypress’ community. I wouldn’t weigh this factor too heavily.
Parallel Test Execution
Playwright runs tests in parallel, Cypress doesn’t. This is a big deal for me. Parallel test executions mean that I can run my test suite MUCH faster. The lack of parallel executions is something I would really hold against Cypress.
Where Playwright is a Better Choice
Playwright is the general better choice. You have a longer learning curve. The advantages of full browser support, better tooling and unified API make it worth it in my opinion. Most companies aim to retain testers for a number of years, so they can make the investment on that initial learning curve.
Where Cypress is a Better Choice
Cypress may be a better choice in a few specific situations. Companies that have high turnover or low technical ability in their testing department could really benefit from Cypress. The quick start time makes it easy to get people up to speed and contributing. Companies who don’t care about cross-browser support or longer test times may also find Cypress more attractive. Cypress isn’t a bad choice, I just think Playwright is better.
Conclusion
Overall Cypress and Playwright both have a lot of offer. A team will be well served by either. In general, I would recommend using Playwright. Playwright has better capabilities, more resources (from Microsoft) and is a more polished product. Cypress is good, but lacks a cohesive structure and fully fleshed out API.
