Markdown API Testing in CI/CD

Overview

Reqbook is a single static binary with stable exit codes, making it straightforward to integrate into any CI pipeline. Common use cases include: validating specs on every pull request, running endpoint tests against staging before a deployment, and detecting secrets committed to spec files before they reach the remote. See the Exit codes reference for a full list of codes and what triggers each one.

Release automation

This repository ships a tag-based release workflow at .github/workflows/release.yml. Push a tag like v0.1.0, or run the release workflow manually with an existing v* tag, to build release binaries, package the VS Code extension, create a GitHub Release, and optionally publish to external package managers. Package-manager publishing is gated by GitHub repository variables. Leave a variable unset or set it to false to keep that channel disabled while still producing GitHub Release artifacts.

GitHub Release artifacts

The release workflow must publish these binary assets because the shell, PowerShell, and npm installers download by exact asset name:

Platform	Required asset
macOS Apple Silicon	`rqb-aarch64-apple-darwin.tar.xz`
macOS Intel	`rqb-x86_64-apple-darwin.tar.xz`
Linux ARM64	`rqb-aarch64-unknown-linux-musl.tar.xz`
Linux x64	`rqb-x86_64-unknown-linux-musl.tar.xz`
Windows x64	`rqb-x86_64-pc-windows-msvc.zip`
VS Code	`reqbook-vscode-<tag>.vsix`

Each CLI archive is published with a matching .sha256 file. The workflow fails before and after creating the GitHub Release if any required asset is missing.

Manual release trigger

Manual release runs are for retrying or promoting an existing tag. Create and push the tag first:

git tag v0.1.0
git push origin v0.1.0

Then open GitHub Actions, choose the release workflow, click Run workflow, and enter:

Input	Value
`tag`	Existing tag, for example `v0.1.0`.
`draft`	`true` only when you want to review the GitHub Release before publishing.
`prerelease`	`true` for alpha, beta, rc, or early public builds.

The workflow checks out the tag, not the current main branch. The tag must start with v.

Required GitHub configuration

Channel	GitHub Secret	Repository Variable	Notes
GitHub Releases	none	none	Uses the built-in `GITHUB_TOKEN` with `contents: write`.
GHCR Docker image	none	none	Uses the built-in `GITHUB_TOKEN` with `packages: write`; publishes `ghcr.io/<owner>/rqb:<tag>` and `latest`.
crates.io	`CARGO_REGISTRY_TOKEN`	`PUBLISH_CRATES=true`	Token must be allowed to publish the `reqbook` crate.
npm	`NPM_TOKEN`	`PUBLISH_NPM=true`	Token must publish the `reqbook` package; workflow uses npm provenance.
Visual Studio Marketplace	`VSCE_PAT`	`PUBLISH_VSCODE=true`	PAT must manage the `reqbook` publisher.
Open VSX	`OVSX_PAT`	`PUBLISH_OPEN_VSX=true`	Token must publish under the manifest publisher namespace.
Homebrew tap	`HOMEBREW_TAP_TOKEN`	`PUBLISH_HOMEBREW=true`	Reserved for a tap update job; this repository does not update a tap yet.

Optional Homebrew variables for a future tap job:

Variable	Example	Purpose
`HOMEBREW_TAP_REPOSITORY`	`ngoclinh93qt/homebrew-tap`	Target tap repository.
`HOMEBREW_FORMULA_NAME`	`rqb`	Formula file/name to update.

Setting secrets and variables

gh secret set CARGO_REGISTRY_TOKEN
gh secret set NPM_TOKEN
gh secret set VSCE_PAT

gh variable set PUBLISH_CRATES --body true
gh variable set PUBLISH_NPM --body true
gh variable set PUBLISH_VSCODE --body true

Open VSX and Homebrew can stay disabled until those channels are ready:

gh variable set PUBLISH_OPEN_VSX --body false
gh variable set PUBLISH_HOMEBREW --body false

Do not store RQB_* runtime API credentials as release publishing secrets unless a CI test needs them. RQB_* values are for executing API specs, while the package-manager tokens above are for publishing artifacts.

GitHub Actions

Install Reqbook

Add a step to download the binary. The install script detects the platform automatically.

- name: Install Reqbook
run: curl -fsSL https://markapidown.net/install.sh | sh

Validate specs

Run rqb validate against the entire api-docs/ directory. Exit code 2 means a spec has a structural error; exit code 5 means a secret was committed.

- name: Validate specs
run: rqb validate api-docs/

Run endpoint tests

Pass secrets via RQB_* environment variables. They map automatically to camel-case variable names in your specs.

- name: Run endpoint tests
env:
RQB_AUTH_TOKEN: ${{ secrets.STAGING_AUTH_TOKEN }}
RQB_BASE_URL: https://staging.example.com
run: |
rqb exec api-docs/apis/health/get-health.md --env=staging
rqb exec api-docs/apis/users/get-users.md --env=staging

Run pipelines

Use rqb flow for multi-step scenarios that chain request captures across endpoints.

- name: Run onboarding pipeline
env:
RQB_AUTH_TOKEN: ${{ secrets.STAGING_AUTH_TOKEN }}
run: rqb flow api-docs/flows/user-onboarding.md --env=staging

Full workflow example

name: API spec tests

on:
 push:
 branches: [main]
 pull_request:
 paths:
 - 'api-docs/**'

jobs:
 test:
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4

 - name: Install Reqbook
 run: curl -fsSL https://markapidown.net/install.sh | sh

 - name: Validate specs
 run: rqb validate api-docs/

 - name: Run endpoint tests
 env:
 RQB_AUTH_TOKEN: ${{ secrets.STAGING_AUTH_TOKEN }}
 RQB_BASE_URL: https://staging.example.com
 run: |
 rqb exec api-docs/apis/health/get-health.md --env=staging
 rqb exec api-docs/apis/users/get-users.md --env=staging

 - name: Run onboarding pipeline
 env:
 RQB_AUTH_TOKEN: ${{ secrets.STAGING_AUTH_TOKEN }}
 run: rqb flow api-docs/flows/user-onboarding.md --env=staging

JUnit reports

Generate JUnit XML output for CI test reporters. Most CI systems can parse JUnit XML to display per-test results inline in pull request checks.

rqb exec api-docs/apis/users/get-user-by-id.md \
 --env=staging \
 --output=junit > results.xml

Upload and publish in GitHub Actions:

- name: Run tests and capture output
 env:
 RQB_AUTH_TOKEN: ${{ secrets.STAGING_AUTH_TOKEN }}
 run: |
 rqb exec api-docs/apis/users/get-user-by-id.md \
 --env=staging --output=junit > results.xml

- name: Upload test results
 uses: actions/upload-artifact@v4
 with:
 name: rqb-results
 path: results.xml

- name: Publish test report
 uses: dorny/test-reporter@v1
 with:
 name: Reqbook API tests
 path: results.xml
 reporter: java-junit

Passing secrets safely

Use RQB_* environment variables to inject secrets. The RQB_ prefix is stripped and the name is converted to lower camel case before variable resolution.

env:
 RQB_AUTH_TOKEN: ${{ secrets.API_TOKEN }} # becomes {{authToken}}
 RQB_STRIPE_KEY: ${{ secrets.STRIPE_KEY }} # becomes {{stripeKey}}

Do NOT use --var authToken=$SECRET in CI. Shell argument lists may appear in CI logs, exposing the secret value. Always use RQB_* environment variables to pass secrets into Reqbook.

Exit code handling

rqb exec api-docs/apis/users/get-user.md --env=staging
CODE=$?
case $CODE in
 0) echo "Passed" ;;
 1) echo "Response mismatch check the spec or API behavior" ; exit 1 ;;
 4) echo "Network error check VPN, service health, or BASE_URL" ; exit 1 ;;
 5) echo "Secret committed to spec file fix before merging" ; exit 1 ;;
 *) echo "Unexpected error (code $CODE)" ; exit 1 ;;
esac

Treat exit code 4 (network error) differently from exit code 1 (response mismatch) in your CI pipeline. A network error means the staging environment may be down it does not necessarily mean your spec is wrong.

GitLab CI

stages:
 - validate
 - test

validate-specs:
 stage: validate
 script:
 - curl -fsSL https://markapidown.net/install.sh | sh
 - rqb validate api-docs/

api-tests:
 stage: test
 variables:
 RQB_AUTH_TOKEN: $STAGING_AUTH_TOKEN
 RQB_BASE_URL: https://staging.example.com
 script:
 - rqb exec api-docs/apis/health/get-health.md --env=staging
 - rqb flow api-docs/flows/user-onboarding.md --env=staging
 only:
 - main
 - merge_requests

Pinning the binary version

For reproducible CI builds, pin the Reqbook version explicitly rather than always installing the latest release.

# Install a specific version
curl -fsSL https://markapidown.net/install.sh | sh -s -- --version=v1.2.0

Alternatively, check the binary into your repository under bin/rqb and reference it directly. The binary is statically linked and has no runtime dependencies.

Get started

API specs

Guides

Markdown API Testing in CI/CD

Overview

Release automation

GitHub Release artifacts

Manual release trigger

Required GitHub configuration

Setting secrets and variables

GitHub Actions

Full workflow example

JUnit reports

Passing secrets safely

Exit code handling

GitLab CI

Pinning the binary version

​Overview

​Release automation

​GitHub Release artifacts

​Manual release trigger

​Required GitHub configuration

​Setting secrets and variables

​GitHub Actions

​Full workflow example

​JUnit reports

​Passing secrets safely

​Exit code handling

​GitLab CI

​Pinning the binary version

Overview

Release automation

GitHub Release artifacts

Manual release trigger

Required GitHub configuration

Setting secrets and variables

GitHub Actions

Full workflow example

JUnit reports

Passing secrets safely

Exit code handling

GitLab CI

Pinning the binary version