How to compare two OpenAPI versions without missing a breaking change
· The Vizroute team
Why a text diff and even a JSON diff miss the changes that break API clients, what an API-aware comparison checks instead, and a short review checklist.
A pull request changes openapi.yaml. The diff shows 412 lines. Somewhere in there a field went from optional to required, and the mobile app that sends requests without it will start getting 400s on the day this ships. Nobody saw it in review, because nobody can read 412 lines of YAML and hold the whole API in their head while they do it.
Comparing two versions of an API is a different job from comparing two files. This post goes through why the usual tools miss the changes that matter, and what to check instead.
A text diff answers the wrong question
git diff compares lines. An OpenAPI document is not a list of lines, it is a tree, and most edits to a tree move lines around without changing what the API does:
- Reformatting, or converting the file from JSON to YAML, changes every line and nothing else.
- Moving a path to another place in the file shows up as a deletion and an addition.
- Pulling a schema out into
componentsand replacing it with a$reflooks like a large change. To a client it is no change at all.
The opposite failure is worse. Adding required: [customerId] is one line, easy to scroll past among hundreds, and it can break every client that does not send that field.
A JSON diff is closer, but it does not know what an operation is
A structural diff, such as the JSON diff tool, compares the tree instead of the text. Key order and formatting stop counting, which removes most of the noise.
What it still cannot tell you is what a change means. It will report that paths./orders.get.parameters[2] was added. Whether that is a new optional filter, which is harmless, or a new required header, which breaks every caller, depends on knowing that the thing under parameters is a parameter and what required does to a client. A document diff does not know either.
What an API-aware comparison checks
Comparing as an API means matching operations, not lines. Each endpoint in the old version is paired with the endpoint at the same method and path in the new one. Then, for each pair, the parts a client depends on are compared one by one:
| Part | What counts as a change |
|---|---|
| Parameters | Added, removed, retyped, or switched between optional and required, matched by location and name |
| Request body | Appearing or disappearing, and fields added, removed, retyped or made required, including nested fields |
| Responses | Status codes added or removed, and response fields added, removed or retyped for each status |
| Lifecycle | An operation newly marked deprecated |
| Documentation | A new summary or description |
$refs are followed before anything is compared, so moving a schema into components reports nothing. Key order and formatting report nothing. What is left is the list of changes a client could actually notice.
Which changes break clients
Not every change is a problem. The rough rule is that taking something away, or demanding something new, breaks callers. Offering something new does not.
| Change | Effect on existing clients |
|---|---|
| An endpoint removed | Breaking. Every call to it fails |
| A new required parameter or body field | Breaking. Old requests are now invalid |
| An optional parameter made required | Breaking, for the same reason |
| A response field removed | Breaking for any client that reads it |
| A response field's type changed | Breaking. "id": "42" and "id": 42 parse differently |
| A parameter removed | Usually safe for clients, but worth a warning. Callers still sending it may be relying on it doing something |
| An endpoint deprecated | Safe today. It is a promise to break it later |
| A response field added | Safe for well-behaved clients. Strict parsers that reject unknown fields will still complain |
| A new endpoint | Safe |
A review checklist
When an OpenAPI file changes in a pull request, these five questions catch most of the damage:
- Did any endpoint disappear, or change its path? A renamed path is a removal plus an addition.
- Is anything newly required, in the parameters or in the request body?
- Did any response field disappear or change type?
- Did a success status change, for example from 200 to 201, or from 200 to 204 with no body?
- Is anything newly deprecated, and is there a date and a replacement in the description?
Doing it in Vizroute
Vizroute has both comparisons, and they answer different questions.
Breaking changes grades each difference as Breaking, Warning, Info or Safe. It covers removed endpoints, new required parameters, removed response fields, changed field types, removed parameters, deprecations, new response fields and new endpoints. It can export the result as a Markdown report or copy it as a pull request comment. It does not yet look at request bodies, or at an existing parameter that became required.
API diff lists every changed endpoint with each change underneath it, request bodies and required-flag changes included. Use it to see everything that moved, and Breaking changes to decide whether you can ship it.
Both run in the browser, on two files you paste or pick, and the specifications are not uploaded. Open either from the command palette in the API Map (press Ctrl+K), or use the Check a change card on the Home page. The user guide describes both in more detail.
Two limits are worth knowing. Endpoints are matched by method and path, so a renamed path shows as one endpoint removed and another added. And a Postman collection declares no response schemas, so responses are only compared for OpenAPI and Swagger files.