Playwright End-to-End Testing Framework (UI+API) with Azure DevOps Pipeline

Main topics covered:

- Page Object Model Design Pattern

- UI Automation - Web Application

- API Testing - REST API's

- Azure DevOps Pipeline - CI CD with YAML or without YAML File

- Visual Studio Code

• What is Playwright? and Advantages, Limitations?

Playwright is an open-source automation testing tool which is used test end to end modern web mobile applications in headed or headless mode.

Advantages:

• Cross Browser Testing - Chrome, Edge, Chromium, Firefox & Webkit.
• Cross Platform Testing - Windows, Linux & MacOS.
• Cross Programming Language - JavaScript, TS, Python, .NET & Java.
• Mobile/Web - Mobile emulation of Google Chrome for Android and Mobile Safari.
• Auto Wait
• Tracing
• Reporting
• Dynamic Wait Assertions
• Faster & Reliable
• Powerful Tooling – Codegen, Playwright Inspector & Trace Viewer
• No flaky test

Disadvantages/Limitations:

• Playwright does not have a big community compared to selenium webdriver.
• It does not work on legacy browsers such IE11.
• It is not possible to test with real device mobile apps.

1. Below playwright session covers UI Automation Testing

2. Below playwright session covers API Automation Testing

3. Integrating Playwright with Azure DevOps Pipeline

Step 1: Firstly, Let's start installing Playwright using below command.

npm init playwright@latest        

Step 2: Create all the folders listed in below snapshot.

Step 3: Add the configurations in .env file.

Step 4: Create page object model pages.

This is how sample page object model page looks like

Step 5: Create UI automation test spec file.

// Include playwright module
const {test, expect} = require('@playwright/test');
const { HomePage } = require('../pages/homepage');
const { ResultPage } = require('../pages/resultpage');
const { PlaylistPage } = require('../pages/playlistpage');
import { qaTestData } from "../test-data/qa/google.json";
import { stageTestData } from "../test-data/stage/google.json";

let testData = null;

test.beforeAll('Running before all tests', ()=>{
    if(process.env.ENV == 'qa'){
        testData = qaTestData;
    }else{
        testData = stageTestData;
    }
})

// Write a test
test('Ui automation test in playwright', async({page}) =>{
    // Go to URL
    const homepage = new HomePage(page);
    await homepage.goto();
   
    // Search with keywords
    homepage.searchKeywords(testData.skill1);

    // Click on playlist
    const resultpage = new ResultPage(page);
    resultpage.clickOnPlaylist();

    await page.waitForTimeout(4000);

    // Click on video
    const playlistpage = new PlaylistPage(page);
    playlistpage.clickOnVideo();

    await page.waitForTimeout(8000);
})        

Step 6: Run playwright UI tests.

npx playwright test        

Step 7: Create API automation test spec file.

const { test, expect } = require("@playwright/test");
const postRequest = require("../test-data/api-requests/post_request_body.json");
const tokenRequest = require("../test-data/api-requests/token_request_body.json");
const { json } = require("stream/consumers");

test.use({
  baseURL: process.env.API_BASE_URI.toString()
})

test("End to End api test in playwright", async ({ request }) => {
  // create post api request using playwright
  const postAPIResponse = await request.post("/booking", {
    data: postRequest,
  });

  const bookingId = await postAPIResponse.json();
  const bId = bookingId.bookingid;

  // create GET api request using playwright
  const getAPIResponse = await request.get("/booking/", {
    params: {
      firstname: "testers talk playwright",
      lastname: "testers talk api testing",
    },
  });

  // validate status code
  console.log(await getAPIResponse.json());
  expect(getAPIResponse.ok()).toBeTruthy();
  expect(getAPIResponse.status()).toBe(200);

  // generate token
  const tokenAPIResponse = await request.post("/auth", {
    data: tokenRequest,
  });
  expect(tokenAPIResponse.ok()).toBeTruthy();
  expect(tokenAPIResponse.status()).toBe(200);

  console.log(await tokenAPIResponse.json());
  const tokenResponseBody = await tokenAPIResponse.json();
  const tokenNo = tokenResponseBody.token;

  // partial update booking details
  const patchAPIResponse = await request.patch(`/booking/${bId}`, {
    headers: {
      "Content-Type": "application/json",
      Cookie: `token=${tokenNo}`,
    },
    data: {
      firstname: "testers talk postman",
      lastname: "testers talk rest assured",
    },
  });

  console.log(await patchAPIResponse.json());
  expect(patchAPIResponse.ok()).toBeTruthy();
  expect(patchAPIResponse.status()).toBe(200);

  // DELETE api request
  // partial update booking details
  const deleteAPIResponse = await request.delete(`/booking/${bId}`, {
    headers: {
      "Content-Type": "application/json",
      "Cookie": `token=${tokenNo}`,
    },
  });
  expect(deleteAPIResponse.status()).toBe(201);
  expect(deleteAPIResponse.statusText()).toBe("Created");
});        

Step 8: Run playwright API tests.

npx playwright test        

Step 9: Verify playwright test report in playwright-report folder.

How to Integrate Playwright with Azure DevOps Pipeline?

There are 2 options, option1 is using yaml file & option2 is without using yaml file. let's see one by one.

1. Option1 - Using YAML FileStep1: Create a new project in ADO then Click on Project

Step2: Click on Repos & Let's create new repository, Click on New repository.

Step3: Enter Repository name & Click on Create

Step4: Click on Clone button and get the URL. Go to your system then clone repository.

Step5: Add all the playwright framework folders inside cloned repository.

Step6: Push all the folders into Azure DevOps

Step7: Repository is ready now, let's create pipeline. Click on Pipelines->Create Pipeline

Step8: Click on Azure Repos Git

Step9: Select previously created repository.

Step10: Select Starter Pipeline

Step11: Copy below yaml content and paste it inside azure-pipelines.yml file.

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- task: NodeTool@0
  inputs:
    versionSpec: '18'
  displayName: 'Install Node.js'
- script: npm ci
  displayName: 'npm ci'
- script: npx playwright install --with-deps
  displayName: 'Install Playwright browsers'
- script: npx playwright test
  displayName: 'Run Playwright tests'
  env:
    CI: 'true'        

If you are running in self-hosted agent replace pool commands

pool:
   name: AgentPoolName
   demands:
   - agent.name -equals AgentName        

Step12: Click on Save and run

Step13: You will see job queued like this.

Step14: Click on Job & Verify build status.

Step15: Now let's Upload playwright-report folder with Azure Pipelines & Report generation. Firstly update azure-pipelines.yml file

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- task: NodeTool@0
  inputs:
    versionSpec: '18'
  displayName: 'Install Node.js'
- script: npm ci
  displayName: 'npm ci'
- script: npx playwright install --with-deps
  displayName: 'Install Playwright browsers'
- script: npx playwright test
  displayName: 'Run Playwright tests'
  env:
    CI: 'true'

- task: PublishTestResults@2
  displayName: 'Publish test results'
  inputs:
    searchFolder: 'test-results'
    testResultsFormat: 'JUnit'
    testResultsFiles: 'e2e-junit-results.xml'
    mergeTestResults: true
    failTaskOnFailedTests: true
    testRunTitle: 'My End-To-End Tests'
  condition: succeededOrFailed()
- task: PublishPipelineArtifact@1
  inputs:
    targetPath: playwright-report
    artifact: playwright-report
    publishLocation: 'pipeline'
  condition: succeededOrFailed()        

Step16: Verify playwright-report folder attachment & report. From job we can navigate into artifacts folder. Download playwright report and verify results.

Above code can be found in GitHub Repository - BakkappaN/PlaywrightBaseAutomationFramework: Playwright Base Automation Framework (github.com)

Playwright Playlist -

Join to Tester Talk for more updates on latest software testing technologies.

Happy Learning!!