Playwright is an end-to-end testing library developed by Microsoft, allowing you to create and automate tests to ensure web applications function as expected across different web browsers and operating systems.
It supports various programming languages (Typescript, .NET, Java, etc.) and platforms (Windows, Linux, macOS), and can be integrated into CI/CD pipelines, automating the testing process.

Compared to other frameworks, Playwright offers several significant advantages:

Speed and Reliability

Runs tests in parallel across multiple browsers with minimal configuration.
Supports headless mode (without a visible UI) for faster and more efficient execution.

Multi-Browser Support

Compatible with Chromium, Firefox, WebKit, and provides features for mobile testing.

Advanced Automation

Simulates complex interactions such as drag-and-drop, file handling, and network behavior.
Supports API testing, extending automation beyond the UI.

Resilient Testing

Implements an auto-retry mechanism, reducing test failures due to slow page loads.
Supports advanced waiting conditions, avoiding the need for sleep() and improving test stability.

Easy Debugging

Captures screenshots and video recordings of test executions, making error analysis more efficient.
Includes an interactive debug mode, allowing developers to pause test execution and inspect the application’s state in real time.

CI/CD Integration

Seamlessly integrates with continuous integration tools like GitHub Actions, Jenkins, and GitLab CI, enabling automated testing for every deployment.

In this article, we will use the library with Typescript to test a web application, utilizing the dedicated Playwright plugin available in the VS Code store.

Tool Features

Step Recording

You can start recording a test and its steps by selecting Testing > Record New, where the recording of actions will begin as you navigate through the application.

Picture 1: section Testing VS Code

At this point, Playwright’s CodeGen will be launched, supported by the VS Code plugin, to simplify the automated creation of tests. This tool offers several features, including:

An interactive browser that automatically records the actions performed by the user.
A test file generated in VS Code, containing the instructions corresponding to the recorded actions.
A toolbar integrated into the browser, useful for analyzing the elements of the web application.

Among the most notable features of the toolbar, we highlight:

Pick-locator: allows you to select an element on the page and generate a JavaScript instruction in VS Code to reference it in the tests.
Assert (visibility, text, value): enables the automatic creation of a JavaScript instruction in VS Code to check specific properties of an element, such as visibility, text, or value.

Figure 2: View with CodeGen running.

To stop the recording and close the CodeGen tool, you can use the recording button available on the toolbar.

Figure 3: Toolbar and recording button.

Running a test

To start running a test, you can use:

The graphical interface provided by the VS Code plugin, which simplifies test execution and debugging.
The Playwright CLI, for greater control and integration into automation processes.

Within the execution settings section, you can configure specific parameters to customize the behavior of the tests, optimizing the testing environment based on the project’s needs.

Figure 4: Playwright settings in VS Code.

Projects: allows you to select the browsers on which the tests will be executed.
Settings: lets you configure the execution by choosing from different modes:
- - Headed: displays the browser during execution, allowing you to observe actions in real time.
  - Trace Viewer: enables an advanced tool to analyze the actions performed in detail.
  - Headless: runs the tests in silent mode, without opening the browser.
Tools: provides additional tools for analyzing and debugging the tests.
- - Pick Locator: allows you to identify and select an element within the web application, as seen during the recording phase.
  - Record New: starts recording a new test.
  - Record at Cursor: records actions in the active file, inserting them starting from the cursor position.
  - Reveal Test Output: opens the page containing the results of the last test execution.

Figure 5: Test explorer in VS Code.

You can use the Explorer section in VS Code to run tests, with the option to enable debug mode and set breakpoints. This allows you to execute the code step by step, monitor the execution, and analyze the variables in real-time.

Figure 6: Execution buttons on the test.

Right-clicking on the buttons displays additional options, such as the ability to start debug mode.

Figure 7: Example of execution in VS Code.

Test automation with GitLab CI/CD.

Tests can be integrated within the repository of a web app. In combination with deploying to a designated environment, it is possible to create a job that automatically runs the available tests on the endpoint.

  # Variables  
  variables:  

    BASE_URL: $BASE_URL # variable defined in GitLab Project CI/CD Settings  
    # Define the stages for your pipeline  
  stages:  
    - test  
  
  # Define the job for running Playwright tests  
  run_playwright_tests:  
    stage: test  
    image: mcr.microsoft.com/playwright:v1.39.0-jammy  

    script:  
      - npm ci # Install project dependencies  
      - npx playwright test # Run your Playwright tests   
      - echo "https://$CI_PROJECT_NAMESPACE.gitlab.io/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/playwright-report/index.html" # print the URL to view the results  

    allow_failure: true   

# Publish Playwright test results as artifacts and keep for 2 days  

  artifacts:  
    when: always  
    paths:  
      - playwright-report  
    expire_in: 2 days

Custom result reporting with the Reporter feature.

Playwright provides the option to implement a custom Reporter, different from the default one, for advanced management of test step results. The Reporter is a component of the library that receives all the data related to the test execution, allowing for custom analysis and reporting.

  import { FullConfig, FullResult, Reporter, Suite, TestCase, TestError, TestResult, TestStep } from "playwright/types/testReporter";

  class TestResultReporter implements Reporter {
    onBegin(config: FullConfig, suite: Suite): void {
    }
    onEnd(result: FullResult): Promise<{ status?: FullResult["status"]; } | undefined | void> | void {
    }
    onError(error: TestError): void {
    }
    onExit(): Promise<void> {
      return new Promise(() => {});
    }
    onStdErr(chunk: string | Buffer, test: void | TestCase, result: void | TestResult): void {
    }
    onStdOut(chunk: string | Buffer, test: void | TestCase, result: void | TestResult): void {
    }
    onStepBegin(test: TestCase, result: TestResult, step: TestStep): void {
    }
    onStepEnd(test: TestCase, result: TestResult, step: TestStep): void {
    }
    onTestBegin(test: TestCase, result: TestResult): void {
    }
    onTestEnd(test: TestCase, result: TestResult): void {
    }

    printsToStdio(): boolean {
      return true;
    }
  }

The Reporter can then be used by specifying it in the playwright.config.ts configuration file, where you can configure the desired reporting type and integrate it with other custom functionalities.

import { defineConfig } from '@playwright/test';
export default defineConfig(
{  
   reporter: './my-awesome-reporter.ts',
 }
);

For example, within the various callbacks, it is possible to send the results directly to a database via queries, allowing for centralized management and easy consultation of test data.

  onTestEnd(test: TestCase, result: TestResult): void {
    const connection = await mysql.createConnection(dbConfig);  // Crea una connessione al database
      const testCaseResult = await connection.execute(
  `INSERT INTO test_cases (test_case_id, test_description, test_results, browser_name, execution_time, execution_error, execution_date, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,

      [
          test.id || null,
          test.title || null,
          test.results,
          test.browserName || null,
          result.duration,
          result.error || null,
          new Date(),
          new Date(),
          new Date()
        ]
      );
    ...
  }

Conclusion

The tool has been successfully employed in several testing use cases, including:

Ensuring the non-regression of existing functionalities when introducing new features.
Running reliability tests over time through the use of cyclic tests.
Supporting the development of new features following a test-driven approach.

These applications have allowed us to improve test reliability, reduce verification times, and effectively simulate real-world usage scenarios. The tool has proven particularly useful in ensuring the quality of complex web applications, characterized by intricate user interactions.

Introduction to Playwright