Technical

Smoke Test your API with Postman API, Newman and GitHub actions.

Juan Tenorio

• 5 min read
Smoke Test your API with Postman API, Newman and GitHub actions.
Photo by Thomas Stephan / Unsplash

I usually do smoke testing on my APIs when I deploy them.

When using Azure and regular Web Apps or Functions, there is something called slots.

Slots are useful when you do a deployment slot, perform basic smoke testing and if it is successful, you just switch the slot to production.

When using containers, there is an option for A/B testing, but basically it is the same concept, if successful direct traffic to the newly deployed app.

But how to automate this testing in your CI/CD pipeline? Well, you can use your postman collections and environments directly from your pipeline and expect results.

Postman API

First, we will need to create an API key in order to access the functionality.

On your Postman desktop application, click on your profile and select settings.

Create a new key.

Copy the key and we will create a new collection and environment in Postman.

In my case, I am creating a collection named Postman API and setting the authorization to API Key. Note that the key name should be X-Api-Key.

Finally paste the key you obtained from Postman.

Now for the environment

The variables that we will be using are

All the variables except postmanapi we will obtain from the API, this is necessary to obtain the GUIDS that we will need in GitHub Actions.

Next, we will get the GUIDS. In order we will get

  1. All workspaces
  2. Get specific workspace
  3. Get specific collection
  4. Get specific environment

All workspaces request

GET {{postmanapi}}/workspaces

You will get a response similar to this, copy the id of the workspace you want to use and place it on the proper variable (workspace) in the environment.

GET {{postmanapi}}/workspaces/{{workspace}} request

As an example, I am interested on the collection from a previous post

https://darthseldon.net/generate-your-own-qr-codes-with-a-minimal-api/

The collection only calls the endpoint and generates an image

And the environment is just the URL

So, we need to copy the id for the collection QR Code and environment Darth Seldon QR Code Azure respectively.

Newman

What is Newman?

"Newman is a command-line Collection Runner for Postman. It enables you to run and test a Postman Collection directly from the command line. It's built with extensibility in mind so that you can integrate it with your continuous integration (CI) servers and build systems." - Postman Learning Center

Run and test collections from the command line using Newman CLI | Postman Learning Center
Postman is a collaboration platform for API development. Postman’s features simplify each step of building an API and streamline collaboration so you can create better APIs—faster.
Postman Learning Center

We can incorporate the tool on GitHub actions, but first we need to create 1 secret and 2 variables on the pipeline.

Go to the settings on your repo and select secrets and variables.

Add the secret

The value is your Postman API key.

Next add the following variables

The values are the values you obtained from the Postman API calls.

GitHub Actions

smoketest:
    runs-on: ubuntu-latest    
    needs: deploy
    steps:
      - name: installnewman
        run: npm install newman -g

      - name: installnewmanreporter        
        run: npm install newman-reporter-junitfull -g

      - name: runnewman        
        run: newman run "https://api.getpostman.com/collections/${{ vars.COLLECTIONID }}?apikey=${{ secrets.POSTMANAPIKEY }}" --environment "https://api.getpostman.com/environments/${{ vars.ENVIRONMENTID }}?apikey=${{ secrets.POSTMANAPIKEY }}" --timeout-request 60000
        
      - if: failure()
        run: curl -X POST -d 'Smoke test failed' -k https://myntfyserver/ntfy/builds

First, we will be using an ubuntu runner and install Newman with NPM.

Second, we will be installing the Newman Reporter.

Third, we will execute the collection along with the environment, note that we are passing the variables and secret and we will be getting the definition directly from the API, so cool!!!

Finally, if Newman execution fails, we will be sending a NTFY alert to the phone, you can skip this step.

Just for fun, I will place the code for the whole YAML file, which builds, deploys and performs the smoke test.

# Docs for the Azure Web Apps Deploy action: https://github.com/Azure/webapps-deploy
# More GitHub Actions for Azure: https://github.com/Azure/actions

name: Build and deploy ASP.Net Core app to Azure Web App - darthseldonqrdemo

on:
  push:
    branches:
      - main
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Set up .NET Core
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: '8.x'

      - name: Set up dependency caching for faster builds
        uses: actions/cache@v3
        with:
          path: ~/.nuget/packages
          key: ${{ runner.os }}-nuget-${{ hashFiles('**/packages.lock.json') }}
          restore-keys: |
            ${{ runner.os }}-nuget-
      - name: Build with dotnet
        run: dotnet build src/DarthSeldon.API.QRCode.Demo.sln --configuration Release      

      - name: dotnet publish
        run: dotnet publish src/DarthSeldon.API.QRCode.Demo.csproj -c Release -o ${{env.DOTNET_ROOT}}/myapp

      - name: Upload artifact for deployment job
        uses: actions/upload-artifact@v4
        with:
          name: .net-app
          path: ${{env.DOTNET_ROOT}}/myapp 

  deploy:
    runs-on: ubuntu-latest
    needs: build
    environment:
      name: 'Production'
      url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
    permissions:
      id-token: write #This is required for requesting the JWT

    steps:
      - name: Download artifact from build job
        uses: actions/download-artifact@v4
        with:
          name: .net-app
      
      - name: Login to Azure
        uses: azure/login@v2
        with:
          client-id: ${{ secrets.AZUREAPPSERVICE_CLIENTID }}
          tenant-id: ${{ secrets.AZUREAPPSERVICE_TENANTID }}
          subscription-id: ${{ secrets.AZUREAPPSERVICE_SUBSCRIPTIONID }}

      - name: Deploy to Azure Web App
        id: deploy-to-webapp
        uses: azure/webapps-deploy@v3
        with:
          app-name: 'darthseldonqrdemo'
          slot-name: 'Production'
          package: .
          
  smoketest:
    runs-on: ubuntu-latest    
    needs: deploy
    steps:
      - name: installnewman
        run: npm install newman -g

      - name: installnewmanreporter        
        run: npm install newman-reporter-junitfull -g

      - name: runnewman        
        run: newman run "https://api.getpostman.com/collections/${{ vars.COLLECTIONID }}?apikey=${{ secrets.POSTMANAPIKEY }}" --environment "https://api.getpostman.com/environments/${{ vars.ENVIRONMENTID }}?apikey=${{ secrets.POSTMANAPIKEY }}" --timeout-request 60000
        
      - if: failure()
        run: curl -X POST -d 'Smoke test failed' -k https://myntfyserver/ntfy/builds

Lets check the result.

Failed smoke test

Successful smoke test

When executing the tests you can also do pre-request scripts to obtain tokes to your secured APIs, extract the collection and environment ids, switch the slots in automatic if successful, etc.

I hope you find this useful.

Happy coding!!!