7 Jan 2021
This contribution is a new feature.
Techdocs AWS Support
To understand a little bit more about the initial issue you need to understand what TechDocs is.
Techdocs is a docs-like-code solution build into Backstage. This means you can write your documentation in Markdown files which live next to your code.
Today, it is one of the core products in Spotify’s developer experience offering with 2,400+ documentation sites and 1,000+ engineers using it daily. backstage.io
To render the documentation, TechDocs uses the generated static files.
In the "recommended" setup you need to add a cloud storage providers like
AWS S3, etc.
Google GCS is supported by TechDocs, the goal of the issue is to implement
AWS S3 as TechDocs external storage to store and read generated documentation sites.
The code blocks are intentionally incomplete for the sake of readability.
If you want to read the full code you'll find it in the PR link at the top.
Add a new
The publisher is used for two things:
- publish generated static files to a storage
- read files from storage when users visit a TechDocs page
Each publisher needs to implement
PublisherBase abstract class and its four methods (
publishmethod is used to upload all the files from the generated
directoryto the S3 bucket.
hasDocsBeenGeneratedmethod is used to check if index.html of an Entity's docs site is available.
fetchTechDocsMetadatamethod is used to fetch the
techdocs_metadata.jsonfile from our bucket.
docsRoutermethod is used to create an express middleware that serves static files in techdocs-backend.
Before implementaing our methods we need to instantiate the AWS SDK with some config:
credentials.accessKeyId: the User access key id
credentials.secretAccessKey: the User secret access key
region: AWS Region
Now that our sdk is instantiated we can implement our methods.
We'll take the example of the
Add tests and mock
I followed the TDD method by writing my tests first and then write the code that will allow these tests to pass.
For a better understanding of this article, I prefer to present the code to you before presenting the tests.
Following the BDD Approach for the
fetchTechDocsMetadata test, we have something like:
- Given a
- When the user wants to fetch the tech docs metadata
- Then the user gets the tech docs metadata content
By taking the previous method, we have a test file that looks like:
As you can see in the code above, we don't actually use the real AWS SDK, we mocked it.
To test the publisher behavior, we need to mock the AWS SDK which provides a JS API for AWS services.
To do this I used jest's mock feature. As our library is called
aws-sdk, we will create a file
__mocks__ containing our implementation of the
We will then have to define in this file an
S3 class which corresponds to the class we are using.
For the tests we mock the reading of our files from a bucket with local files that we mock with
The main step here was to explain to the users how they can configure an AWS S3 Bucket with TechDocs.
As specified in the comments of the pull request, a next feature will be implemented on top of this one to handle S3 configuration apart from creating an access user agent.
It will add the possibility to read from the instance profile or
This step contains all the other elements that form the glue between the main pieces of this contribution.
I still find it important to add it since without these elements our code cannot work.
Our job as developers is also to write documentation, add comments ... which will improve the Developer experience (DX).
The final step is to add changesets which will contains the list of our file changes.
It lets us declare how our changes should be released.
In our case we only have
Someone in the comments suggested using as it has first-class TypeScript support.
The issue was that there was a problem with Typescript that was going to be fixed in a .
So I had to wait until the fix was merged to bump the aws sdk version.
This PR also made it possible to identify new features:
- Enable publishers support using nested directories in a bucket:
- Load GCS credentials from the environment:
- Load AWS credentials from the shared credentials file:
This contribution has allowed me to use the
aws-sdk v3 and to compare it with the v2 version.
It also allowed me to improve my english by writing some documentation (Not being a native English speaker, it is important for me to improve myself by practicing my English.)
It allowed me, thanks to the review of the different members working on the project, to improve my code, my logic and to question my work to be more rigorous.