Backstage - Techdocs End-to-End testing
This contribution is a new feature.
Introduction
Project
You can find the Backstage project presentation here.
Context
In this contribution we will talk about a specific part of Backstage: TechDocs.
TechDocs is a docs-like-code plugin that lets you write technical documentation next to your code.
The concept is pretty simple, you write your docs in Markdown files and TechDocs creates a reader-friendly experience for you.
TechDocs consists of a backend plugin (generate, prepare and publish the documentation) and a frontend plugin (renders the documentation to the final user).
We will focus here on the frontend part as this the most relevant to us in this context.
We'll just admit that the backend plugin returns our documentation as HTML/CSS files.
The component of the frontend TechDocs plugin that we will be in charge of testing is called the TechDocs Reader.
The TechDocs Reader will be in charge of getting the HTML file, running transformers on it and then renders it into a shadow DOM root. We will make sure that it does its job properly.
Here is some screenshots of what the frontend plugin looks like inside Backstage.
Current behavior
Some functionality of TechDocs relies on interactions between the BackStage app and the shadow root that contains the TechDocs site.
These interactions should be tested to ensure that the TechDocs features are working properly and avoid regressions.
Here is an example of some e2e tests that we will implement:
- Navigating to a TechDocs site from a given URL
- Navigating to a TechDocs site via the primary navigation bar
- Navigating to a TechDocs site fragment via the table of contents, and so on...
Implement the solution
This PR being still Open, some parts are likely to change.
I will keep the article updated if any changes are made.
To implement our solution we will use Cypress.
But first... What is Cypress?
Cypress is a JavaScript End to End testing framework that lets you write Developer-friendly tests.
Here is a screenshot of the Cypress user interface running Backstage:
In the screenshot above you can see:
- the test status menu used to see how many tests passed or failed
- the app preview used to see what happens in your app while the tests are running
- the command log which shows the different steps of your tests (also called "time travel")
Define custom commands
Cypress comes with its own API for creating custom commands that we can use in our tests.
We will define two commands:
loginAsGuest
to log the User as a guest by setting the custom cookie@backstage/core:SignInPage:provider
toguest
getTechDocsShadowRoot
to get the shadow DOM root of the TechDocs site more easily
Cypress.Commands.add('loginAsGuest', () => {
window.localStorage.setItem('@backstage/core:SignInPage:provider', 'guest');
});
Cypress.Commands.add('getTechDocsShadowRoot', () => {
cy.get('[data-testid="techdocs-content-shadowroot"]').shadow();
});
Configure the viewport
In order to make certain elements visible (like the table of contents), we have to set a custom viewport size.
We will take the macbook-15
preset dimensions and define those values inside the cypress.json
configuration file.
This will tell Cypress to set a custom screen size for our application.
{
"viewportWidth": 1440,
"viewportHeight": 900
}
Add our first tests
Our first test will be to check that the User can correctly access the TechDocs home page.
We can access it by visiting the /docs
endpoint.
it('should navigate to the home TechDocs page', () => {
cy.visit('/docs');
cy.contains('Documentation');
});
Or we can access it through the Backstage context via the primary navigation bar to the left.
Writing the corresponding Cypress tests gives us the following code.
it('should navigate to the TechDocs page via the primary navigation bar', () => {
cy.visit('/');
cy.get('[data-testid="sidebar-root"]')
.get('div')
.get('a[href="/docs"]')
.click();
cy.contains('Documentation');
});
it('should navigate to the TechDocs home page from the "Overview" tab', () => {
cy.visit('/docs');
cy.get('[data-testid="read_docs"]').eq(0).click();
cy.location().should(loc => {
expect(loc.pathname).to.eq('/docs/default/Component/backstage');
});
});
Note that we use the data-testid
selector as by default Cypress will favor these selectors.
By retrieving the elements with a data-testid
attribute, we make sure that our tests are not coupled to the behavior or styling of the element.
It also allows us to show that this element is used within our tests so that everyone is aware.
Once we have selected a specific TechDocs entity, we can check that the User can correctly navigate within the TechDocs pages via the navigation bar to the left.
We will visit the corresponding TechDocs entity page and simulate the clicks on the navigation bar items: Overview > Roadmap
.
it('should navigate to the TechDocs page via the navigation bar', () => {
cy.visit('/docs/default/Component/backstage');
cy.getTechDocsShadowRoot().within(() => {
cy.get('[data-testid="md-nav-overview"]').click();
cy.get('[data-testid="md-nav-roadmap"]').click();
cy.contains('Phases');
cy.contains('Detailed roadmap');
});
});
The User can also navigate within the current page via the table of contents to the right.
By clicking on an anchor link, the page will scroll to the selected item in the page.
To test that we have scrolled to the correct element we will check that the offsetTop
value of our element equals the scrollY
of the window
object.
Here is the Cypress test that covers this case.
it('should navigate to the TechDocs page via the table of contents - Level 1', () => {
cy.visit('/docs/default/Component/backstage/overview/roadmap');
return cy.getTechDocsShadowRoot().within(() => {
cy.get('[data-testid="md-nav-phases"]').click();
cy.get('#phases').then($el => {
cy.window()
.its('scrollY')
.should($scrollY => {
expect($scrollY).to.be.closeTo($el[0].offsetTop, 200);
});
});
});
});
The last test that we want to cover is the Previous/Next
links at the bottom of each page.
We'll check that the Previous
link takes us to the previous page.
Once again we will visit a TechDocs page, click on the previous link defined by its class md-footer-nav__link.md-footer-nav__link--next
and make sure that it takes us to the correct page.
it('should navigate to the next page within a TechDocs page', () => {
cy.visit('/docs/default/Component/backstage/overview/roadmap');
cy.scrollTo('bottom');
cy.getTechDocsShadowRoot().within(() => {
cy.get('.md-footer-nav__link.md-footer-nav__link--next').click();
cy.location().should(loc => {
expect(loc.pathname).to.eq(
'/docs/default/Component/backstage/overview/vision/',
);
});
});
});
Final result
Here is the final test-suite that covers the different interactions between the Backstage context and the TechDocs site embedded. As we can see all the tests are completed in 32s.
Takeaway
Problems encountered
As the TechDocs frontend is strongly linked to the API response and we don't know how all this stuff will change in the future, we will certainly not mock the API response as we used to do but let the backend do its job.
It means that I will certainly need to remove the API mocks in the tests and add data-testid attributes dynamically inside the generated html files.
What did I learn ?
This contribution has allowed me to define some user workflows and use Cypress to test them.