Smoke Test your API with Postman API, Newman and GitHub actions.

I smoke test my APIs on every deploy — cheap insurance against “it built successfully” quietly meaning absolutely nothing about whether the thing actually works.

If you're on Azure Web Apps or Functions, you've got deployment slots: push to a slot, poke it to make sure it's alive, and only then swap it into production, like a bouncer checking ID before letting the new build anywhere near real traffic. Containers get the same idea under a fancier name — A/B testing, blue/green, whatever your team's calling it this quarter. The concept never changes: don't hand real users to something you haven't personally verified isn't on fire.

Naturally, the next question is how to automate that check inside a CI/CD pipeline instead of eyeballing it like an animal. Turns out you can point your pipeline straight at your own Postman collections and environments and just ask them for a verdict.

Postman API

First step: an API key. In the Postman desktop app, profile → Settings → API keys, generate a new one, and guard it like it's a house key, not a fun little string — it's going straight into a GitHub secret shortly and has real access to your stuff.

Next, build a small collection to smoke test against, authorization set to API Key, key name specifically X-Api-Key (Postman is oddly particular about this, don't ask me why, ask them), then paste in the key you just generated. Alongside it, set up an environment holding the handful of variables the whole setup leans on. Every one of those variables except postmanapi itself gets its value from Postman's own API — meaning the next step is calling that API just to go fetch the values that describe the API calls you're about to make. Delightfully recursive.

In order, that's:

1. Get all workspaces
2. Get a specific workspace
3. Get a specific collection
4. Get a specific environment

Starting with all workspaces:

GET {{postmanapi}}/workspaces

That returns a list of your workspaces — grab the id of the one you actually care about and drop it into the workspace variable. Resist the urge to grab a random one just to see what happens. I have not resisted this urge personally, results varied.

Then, drilling into that specific workspace:

GET {{postmanapi}}/workspaces/{{workspace}}

For this walkthrough, I'm targeting the collection from the QR Codes post — a deliberately tiny collection that just hits the endpoint and confirms it returns an image, and doesn't try to be clever about it, plus an environment holding nothing more than the deployed URL. From that workspace response, grab the ids for the QR Code collection and the Darth Seldon QR Code Azure environment — those two GUIDs are the actual payoff of this entire section, and they're what the GitHub Actions workflow needs next. Everything up to now was just an elaborate scavenger hunt for two strings of hexadecimal.

Newman

What's Newman? No, not the “Hello, Newman” one, unfortunately, this one just runs your API tests:

“Newman is a command-line Collection Runner for Postman. It enables you to run and test a Postman Collection directly from the command line. It's built with extensibility in mind so that you can integrate it with your continuous integration (CI) servers and build systems.” — Postman Learning Center

Before wiring it into GitHub Actions, the pipeline needs one secret and two variables. In your repo's Settings → Secrets and variables:

• Secret — POSTMANAPIKEY, set to the same Postman API key from earlier. One key, doing double duty, living its best life.
• Variables — COLLECTIONID and ENVIRONMENTID, set to the two GUIDs from the scavenger hunt above.

GitHub Actions

The smoke test job itself is refreshingly short: install Newman via npm, install a JUnit-flavored reporter for it, then run the collection — fetching both the collection and environment definitions live from Postman's API using the key and IDs above, no local JSON files checked into the repo silently going stale the moment someone edits the collection in the app and forgets to re-export it (we've all been that someone). If the run fails, fire off an NTFY notification straight to your phone, so you find out from your pocket instead of from a customer.

Here's the complete workflow — build, deploy to Azure, then smoke test — pulled straight from the actual pipeline behind the QR Code demo:

# Docs for the Azure Web Apps Deploy action: https://github.com/Azure/webapps-deploy
# More GitHub Actions for Azure: https://github.com/Azure/actions

name: Build and deploy ASP.Net Core app to Azure Web App - darthseldonqrdemo

on:
  push:
    branches:
      - main
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Set up .NET Core
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: '8.x'

      - name: Set up dependency caching for faster builds
        uses: actions/cache@v3
        with:
          path: ~/.nuget/packages
          key: ${{ runner.os }}-nuget-${{ hashFiles('**/packages.lock.json') }}
          restore-keys: |
            ${{ runner.os }}-nuget-
      - name: Build with dotnet
        run: dotnet build src/DarthSeldon.API.QRCode.Demo.sln --configuration Release

      - name: dotnet publish
        run: dotnet publish src/DarthSeldon.API.QRCode.Demo.csproj -c Release -o ${{env.DOTNET_ROOT}}/myapp

      - name: Upload artifact for deployment job
        uses: actions/upload-artifact@v4
        with:
          name: .net-app
          path: ${{env.DOTNET_ROOT}}/myapp

  deploy:
    runs-on: ubuntu-latest
    needs: build
    environment:
      name: 'Production'
      url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
    permissions:
      id-token: write #This is required for requesting the JWT

    steps:
      - name: Download artifact from build job
        uses: actions/download-artifact@v4
        with:
          name: .net-app

      - name: Login to Azure
        uses: azure/login@v2
        with:
          client-id: ${{ secrets.AZUREAPPSERVICE_CLIENTID }}
          tenant-id: ${{ secrets.AZUREAPPSERVICE_TENANTID }}
          subscription-id: ${{ secrets.AZUREAPPSERVICE_SUBSCRIPTIONID }}

      - name: Deploy to Azure Web App
        id: deploy-to-webapp
        uses: azure/webapps-deploy@v3
        with:
          app-name: 'darthseldonqrdemo'
          slot-name: 'Production'
          package: .

  smoketest:
    runs-on: ubuntu-latest
    needs: deploy
    steps:
      - name: installnewman
        run: npm install newman -g

      - name: installnewmanreporter
        run: npm install newman-reporter-junitfull -g

      - name: runnewman
        run: newman run "https://api.getpostman.com/collections/${{ vars.COLLECTIONID }}?apikey=${{ secrets.POSTMANAPIKEY }}" --environment "https://api.getpostman.com/environments/${{ vars.ENVIRONMENTID }}?apikey=${{ secrets.POSTMANAPIKEY }}" --timeout-request 60000

      - if: failure()
        run: curl -X POST -d 'Smoke test failed' -k https://myntfyserver/ntfy/builds 

Notice smoketest only kicks off after needs: deploy — there's no dignity in smoke testing an app that never actually made it to production, that's just testing the void.

When it fails, the Actions log shows Newman's assertions pointing directly at whatever broke, plus (if you kept the NTFY step) a notification landing on your phone before you've even finished refreshing the Actions tab like a nervous parent. When it succeeds, it's wonderfully, almost insultingly uneventful — a clean pass, a green checkmark, and absolutely nothing else worth mentioning, which is exactly what a smoke test is supposed to deliver: silence.

From here, there's plenty of room to get fancy: pre-request scripts to mint auth tokens for secured endpoints, scripting the collection/environment ID lookups instead of hunting for GUIDs by hand like it's 2015, or automatically swapping deployment slots the instant the smoke test comes back green, no human required to click the button.

I hope you find this useful. Happy coding!!!