I know the title sound like a clickbait, and I was about to use the F word in it, but I gave up! So, let’s start with some banalities:
- I am often bedeviled by an intense curiosity for a subject. This blog is living proof of that. Responding to this imperative to satisfy this curiosity always leaves me bewildered… What’s the point of investigate all this when in the end you can satisfy to be ignorant and claim it?
- Most of the topics below are explored for simplicity more than exhaustivity. So, I will certainly state the obvious on these subjects 🙂
- It also gave a better understanding of Friedrich Nietzsche’s quote: “Blessed are the sleepy ones: for they shall soon nod off”.
Anyway, my two last top-of-mind professional concerns were:
- The first concern was that I wanted to extend some Cypress tests to performance testing (Frontend performance or Backend performance) more than functional testing
The constrain was simple: “Can I do it with Cypress only and not with a tedious integration of new tool or framework for such performance testing?” As it is p… in the a… to manage several tool Ooops!
Anyway, during this exploration, I had the chance to document:
- load testing principles
- using K6 for load testing
- a quick playwright discovery both with Python and Node
- how to connect lighthouse with Cypress
- how to test Google Analytics with Cypress
This post is a quick roundup of this exploration.
You can find all files on my GitHub account. See https://github.com/bflaven/BlogArticlesExamples/tree/master/cypress_playwright_lighthouse_performance_testing
# You will find these 4 directories cypress_example_1 k6_example_1 playwright_node_example_1 playwright_python_example_1
The genuine cost
Does a tool’s genuine cost should include the time spent (learning curb) to reach tool’s proficiency? My answer is Yes.
For instance, I always try to figure out the time spent to learn a dedicated tool (K6) vs the time spent to tinker an existing and well understood (Cypress) with no success guaranteed. You see the point?
For sure, it’s when you begin to master a tool that you see its limits. As often, thanks to an event (bug, regression, error…), you cruelly realize limits and fragility for your hard-won knowledge!
Thus, I master a little Cypress and I now see that this framework that I use for E2E tests is not really intended to do anything else!
Choosing and investing in a new tool represents a tremendous cost for a team.
In my case, the constraint that applies is once Cypress has been chosen, it must at least be kept. I know that the grass is always greener elsewhere and I know also that novelty is the modern bait. Of course, it would be tempting to choose another tool like K6 or to change the E2E framework once again and choose Playwright, for example, which is the latest test framework I have heard of.
It is true that after reading a few articles; It seems obvious that what I was trying to do was obviously more like load testing or performance testing (Frontend performance or Backend performance) than functional testing!
All my knowledge come from this article! See https://k6.io/docs/testing-guides/load-testing-websites/
Load testing principle
I have grabbed some explanations on K6 website, a very famous loading test framework and so a great source of information.
The main question is “What are the load testing approaches possible?”
The answer is always when you approach the load test, first consider the following perspectives:
- Backend vs. frontend performance
- Protocol-based, browser-based, or hybrid load testing
- Component testing vs. end-to-end testing
So where do you stand? According to me, I don’t know exactly in which category I stand with my need but… from my understanding if I write down a user story, I will say:
“As a PO, I want to test the critical files loading and ensure that these files are “present” in the CMS (Backend) or on the website (Frontend) before I really start to test the functionalities themselves.”
To make it abundantly clear, I add that preferably, loading testing and functional testing will be written with the same framework e.g., Cypress so all actions will be chained in user stories written down in Cypress’ testing files.
Indeed, by analogy, to what is written above, it is overall logical in terms of scheduling. For instance, a lumberjack, before sawing a tree, make sure that the ax is operational, so he will not put the cart before the horse like we said! Capisce? 🤌
Like often, at the same time, by exploring this loading test question, I have explored 2 but secondary testing issues:
- First secondary test is how to make sure that Tag Commander is correctly loaded because all users’ actions are tracked down in Google Analytics.
- Second secondary test I want to leverage on Google’s Lighthouse directly through Cypress.
For those who wonder what Google’s Lighthouse, here the baseline extracted from the official website.
Lighthouse is an open-source, automated tool for improving the quality of web pages. You can run it against any web page, public or requiring authentication. It has audits for performance, accessibility, progressive web apps, SEO, and more.
Few words on Playwright with Node
My idea was once again to measure the qualities of playwright. For the comparison conditions to be identical to test frameworks such as Cypress, CodeceptJS, it is this version of playwright with Node that must be installed. The logic of playwright is then remarkably close to the ones mentioned above.
Few words on playwright with Python
I found much interest in working in Python as it is often the “on-top language” that drove my own automation. By the way as I often say, it is much interesting to learn Python instead of Ruby or Bash because you can thousands of things in Python, since is become a standard for data-science or for Google, the language Go may have my favor one day. For Go is the coming underdog!
- Using Cypress and Google Lighthouse for Performance Testing
- How to Make Writing Performance Tests Easy With Cypress
- Web Performance Testing With Cypress and Google Lighthouse
- request on cypress documentation
- Testing frontend performance with Cypress
- cypress-example-recipes on github
- testing-dom__page-source in cypress-example-recipes on github
- testing-dom__wait-for-resource in cypress-example-recipes on github
- server-communication__request in cypress-example-recipes on github
- A Practical Guide to Intercepting Network Requests in Cypress
- intercept on cypress documentation
- Smart GraphQL Stubbing in Cypress
- Cypress WebSocket intercept
- intercept on runebook.dev
- Improve Your End to End Tests with Cypress Intercept
- Network Requests on docs.cypress.io
- Asserting Network Calls from Cypress Tests
- Analytics and tag testing with Cypress
- Testing — Google Tag Manager with Cypress and other plugins
- Functional testing with Cypress
- Installation in k6.io
- Running k6 in k6.io
- Load testing websites in k6.io
- some k6 example on github
- Welcome to k6-Learn
- Introduction to performance testing with k6
- Load testing websites
- Performance testing with k6
- look for Protocol-based, browser-based, or hybrid load testing
- Awesome List Updates on Mar 09, 2020
- Load Test with K6 and Visualize with InfluxDB and Grafana
- Performance Testing your web app with k6
- How to Use k6 to Run Load Testing for a Website (for free)
- K6 Reporter
- Introduction to Playwright Trace Viewer & Test Generator
- Playwright for Python
- Artillery with Playwright