Skip to main content

Backstage - Techdocs End-to-End testing

Open
14 May 2021
TypescriptCypresse2e testing
Contribution Type

This contribution is a new feature.

Introduction

Contribution presentation
Backstage + TechDocs

Project

You can find the Backstage project presentation here.

Context

In this contribution we will talk about a specific part of Backstage: TechDocs.

TechDocs is a docs-like-code plugin that lets you write technical documentation next to your code.
The concept is pretty simple, you write your docs in Markdown files and TechDocs creates a reader-friendly experience for you.

TechDocs consists of a backend plugin (generate, prepare and publish the documentation) and a frontend plugin (renders the documentation to the final user).
We will focus here on the frontend part as this the most relevant to us in this context. We'll just admit that the backend plugin returns our documentation as HTML/CSS files.

The component of the frontend TechDocs plugin that we will be in charge of testing is called the TechDocs Reader.

The TechDocs Reader will be in charge of getting the HTML file, running transformers on it and then renders it into a shadow DOM root. We will make sure that it does its job properly.

Here is some screenshots of what the frontend plugin looks like inside Backstage.


TechDocs entities
TechDocs Dashboard containing the different documentation entities

TechDocs entitiy
TechDocs documentation entity

Current behavior

Some functionality of TechDocs relies on interactions between the BackStage app and the shadow root that contains the TechDocs site.

Contexts

These interactions should be tested to ensure that the TechDocs features are working properly and avoid regressions.

Here is an example of some e2e tests that we will implement:

  • Navigating to a TechDocs site from a given URL
  • Navigating to a TechDocs site via the primary navigation bar
  • Navigating to a TechDocs site fragment via the table of contents, and so on...

Implement the solution

changes

This PR being still Open, some parts are likely to change.
I will keep the article updated if any changes are made.

To implement our solution we will use Cypress.

But first... What is Cypress?

Cypress is a JavaScript End to End testing framework that lets you write Developer-friendly tests.
Here is a screenshot of the Cypress user interface running Backstage:

Cypress
Cypress presentation

In the screenshot above you can see:

  • the test status menu used to see how many tests passed or failed
  • the app preview used to see what happens in your app while the tests are running
  • the command log which shows the different steps of your tests (also called "time travel")

Define custom commands

Cypress comes with its own API for creating custom commands that we can use in our tests.

We will define two commands:

  • loginAsGuest to log the User as a guest by setting the custom cookie @backstage/core:SignInPage:provider to guest
  • getTechDocsShadowRoot to get the shadow DOM root of the TechDocs site more easily
support/commands.js
Cypress.Commands.add('loginAsGuest', () => {
window.localStorage.setItem('@backstage/core:SignInPage:provider', 'guest');
});

Cypress.Commands.add('getTechDocsShadowRoot', () => {
cy.get('[data-testid="techdocs-content-shadowroot"]').shadow();
});

Configure the viewport

In order to make certain elements visible (like the table of contents), we have to set a custom viewport size. We will take the macbook-15 preset dimensions and define those values inside the cypress.json configuration file.
This will tell Cypress to set a custom screen size for our application.

cypress.json
{
"viewportWidth": 1440,
"viewportHeight": 900
}

Add our first tests

Our first test will be to check that the User can correctly access the TechDocs home page.

We can access it by visiting the /docs endpoint.

it('should navigate to the home TechDocs page', () => {
cy.visit('/docs');
cy.contains('Documentation');
});

Or we can access it through the Backstage context via the primary navigation bar to the left.

Navigating to TechDocs

Writing the corresponding Cypress tests gives us the following code.

it('should navigate to the TechDocs page via the primary navigation bar', () => {
cy.visit('/');
cy.get('[data-testid="sidebar-root"]')
.get('div')
.get('a[href="/docs"]')
.click();

cy.contains('Documentation');
});

it('should navigate to the TechDocs home page from the "Overview" tab', () => {
cy.visit('/docs');
cy.get('[data-testid="read_docs"]').eq(0).click();

cy.location().should(loc => {
expect(loc.pathname).to.eq('/docs/default/Component/backstage');
});
});

Note that we use the data-testid selector as by default Cypress will favor these selectors.
By retrieving the elements with a data-testid attribute, we make sure that our tests are not coupled to the behavior or styling of the element.
It also allows us to show that this element is used within our tests so that everyone is aware.

Once we have selected a specific TechDocs entity, we can check that the User can correctly navigate within the TechDocs pages via the navigation bar to the left.

Navigating bar

We will visit the corresponding TechDocs entity page and simulate the clicks on the navigation bar items: Overview > Roadmap.

it('should navigate to the TechDocs page via the navigation bar', () => {
cy.visit('/docs/default/Component/backstage');

cy.getTechDocsShadowRoot().within(() => {
cy.get('[data-testid="md-nav-overview"]').click();
cy.get('[data-testid="md-nav-roadmap"]').click();

cy.contains('Phases');
cy.contains('Detailed roadmap');
});
});

The User can also navigate within the current page via the table of contents to the right.
By clicking on an anchor link, the page will scroll to the selected item in the page.

Check scroll position

To test that we have scrolled to the correct element we will check that the offsetTop value of our element equals the scrollY of the window object.

Table of content

Here is the Cypress test that covers this case.

it('should navigate to the TechDocs page via the table of contents - Level 1', () => {
cy.visit('/docs/default/Component/backstage/overview/roadmap');

return cy.getTechDocsShadowRoot().within(() => {
cy.get('[data-testid="md-nav-phases"]').click();

cy.get('#phases').then($el => {
cy.window()
.its('scrollY')
.should($scrollY => {
expect($scrollY).to.be.closeTo($el[0].offsetTop, 200);
});
});
});
});

The last test that we want to cover is the Previous/Next links at the bottom of each page.
We'll check that the Previous link takes us to the previous page.

Prev-Next page

Once again we will visit a TechDocs page, click on the previous link defined by its class md-footer-nav__link.md-footer-nav__link--next and make sure that it takes us to the correct page.

it('should navigate to the next page within a TechDocs page', () => {
cy.visit('/docs/default/Component/backstage/overview/roadmap');
cy.scrollTo('bottom');

cy.getTechDocsShadowRoot().within(() => {
cy.get('.md-footer-nav__link.md-footer-nav__link--next').click();

cy.location().should(loc => {
expect(loc.pathname).to.eq(
'/docs/default/Component/backstage/overview/vision/',
);
});
});
});

Final result

Results

Here is the final test-suite that covers the different interactions between the Backstage context and the TechDocs site embedded. As we can see all the tests are completed in 32s.

Contribution presentation
Cypress test-suite

Takeaway

Problems encountered

As the TechDocs frontend is strongly linked to the API response and we don't know how all this stuff will change in the future, we will certainly not mock the API response as we used to do but let the backend do its job.
It means that I will certainly need to remove the API mocks in the tests and add data-testid attributes dynamically inside the generated html files.

What did I learn ?

This contribution has allowed me to define some user workflows and use Cypress to test them.