MarkApiDown Exit Codes Reference for CI and Scripting

Every mad command exits with a well-defined numeric code. These codes are stable across releases, which means you can safely branch on them in shell scripts, Makefile targets, and CI pipeline steps without worrying about them changing between versions. The table below covers all six codes, what triggers each one, and how to handle them reliably.

Exit code reference

Code	Meaning	When it occurs
`0`	Passed	All specs are valid and all assertions passed. The actual response matched the `## Expected response` block.
`1`	Test failed	The request was sent and a response was received, but one or more assertions or expected-response comparisons did not match.
`2`	Invalid spec	The spec file is malformed — bad frontmatter, a missing required section, an unresolved `{{variable}}` reference, or an `--env` value with no matching heading in `env.md`.
`3`	Engine error	An internal execution error prevented MarkApiDown from building the HTTP request (unsupported protocol, filesystem error, corrupted binary).
`4`	Network error	The request was built and dispatched but no response arrived — DNS failure, connection refused, or timeout exhausted across all retry attempts.
`5`	Secret detected	A value matching a known secret pattern was found in a versioned markdown file. `mad` exits immediately before any network request is made.

When each code fires in detail

Code 1 — Test failed The request succeeded at the transport level, but the response diverged from the spec. Check the diff output (or use --output=json to parse the assertions array) to see exactly which fields did not match. Code 2 — Invalid spec MarkApiDown could not construct a valid request from the file. The error message always includes the file path, the line number when available, and a suggested fix. Common causes: typo in frontmatter, a {{variable}} with no definition in any environment file or .env.local, or a pipeline step referencing a non-existent endpoint file. Code 3 — Engine error An internal failure occurred after the spec was parsed but before the request was sent. Common causes: an unsupported value for the protocol frontmatter key, a filesystem error while reading an included file, or a missing feature compiled out of the binary. Run mad --verbose for full stack context. Code 4 — Network error MarkApiDown reached the send stage but got no response. This covers DNS resolution failure, refused connections, and timeout expiry across all configured retry attempts. Check VPN status, service health, or whether baseUrl is set correctly in your environment file. Code 5 — Secret detected mad scans spec content for patterns matching JWTs, hex API keys, sk_-prefixed keys, and similar secrets before making any network call. When a match is found the process exits code 5 immediately. Move the value to .env.local (which is auto-added to .gitignore by mad init) or export it as a MAD_* environment variable.

Using exit codes in shell scripts

Test the exit code directly in an if statement to branch on success or failure:

if mad exec api-docs/apis/health/get-health.md --env=staging; then
  echo "Health check passed"
else
  echo "Health check failed — see diff above"
  exit 1
fi

Capture the code in $? to distinguish between failure types:

mad exec api-docs/apis/users/get-user.md --env=staging --var id=usr_123
CODE=$?

if [ $CODE -eq 1 ]; then
  echo "Response mismatch — check the spec or API behaviour"
  exit 1
fi
if [ $CODE -eq 4 ]; then
  echo "Network unreachable — check VPN or service health"
  exit 1
fi
if [ $CODE -ne 0 ]; then
  echo "Unexpected error (code $CODE)"
  exit 1
fi

Use a case statement to handle every code explicitly:

mad exec api-docs/apis/users/get-user.md --env=staging
CODE=$?

case $CODE in
  0) echo "Passed" ;;
  1) echo "Response mismatch — check the spec or API behaviour" ; exit 1 ;;
  2) echo "Invalid spec — run mad validate api-docs/ to see errors" ; exit 1 ;;
  3) echo "Engine error — run mad --verbose for details" ; exit 1 ;;
  4) echo "Network error — check VPN, service health, or baseUrl" ; exit 1 ;;
  5) echo "Secret detected in spec — move it to .env.local before merging" ; exit 1 ;;
  *) echo "Unexpected error (code $CODE)" ; exit 1 ;;
esac

Using exit codes in Makefile targets

Integrate mad into your build pipeline using standard Makefile idioms. The .PHONY declaration prevents Make from confusing targets with files of the same name.

.PHONY: validate test-api

validate:
	mad validate api-docs/

test-api: validate
	mad exec api-docs/apis/health/get-health.md --env=staging
	mad flow api-docs/flows/smoke-test.md --env=staging --output=junit > test-results/flow.xml

To fail the Makefile run on any non-zero exit code (the default behaviour) just call mad as a recipe line. Make stops immediately if a recipe command returns non-zero.

ci: validate test-api
	@echo "All API checks passed"

Using exit codes in GitHub Actions

GitHub Actions fails a step whenever the command exits non-zero, so a bare mad exec call is enough to fail the job on a test failure. Use the steps below to also capture a JUnit report as a downloadable artifact and a test-summary annotation.

name: API spec tests

on:
  push:
    branches: [main]
  pull_request:
    paths:
      - 'api-docs/**'

jobs:
  api-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install MarkApiDown
        run: curl -fsSL https://markapidown.net/install.sh | sh

      - name: Validate all specs
        run: mad validate api-docs/

      - name: Run smoke-test pipeline
        run: mad flow api-docs/flows/smoke-test.md --env=staging --output=junit > test-results.xml
        env:
          MAD_BASE_URL: ${{ secrets.STAGING_BASE_URL }}
          MAD_AUTH_TOKEN: ${{ secrets.STAGING_TOKEN }}

      - name: Upload test report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: api-test-results
          path: test-results.xml

Use if: always() on the upload step so the artifact is saved even when the mad flow step fails. Without it, a failing test deletes the evidence you need to debug it.

When you need to parse the reason for a failure programmatically — for example to post a custom comment on a pull request — run with --output=json. The JSON result object includes a top-level passed boolean, an assertions array with per-assertion outcomes, and duration_ms. See Output Formats for the full schema.

Documentation Index

​Exit code reference

​When each code fires in detail

​Using exit codes in shell scripts

​Using exit codes in Makefile targets

​Using exit codes in GitHub Actions

Exit code reference

When each code fires in detail

Using exit codes in shell scripts

Using exit codes in Makefile targets

Using exit codes in GitHub Actions