What visual bugs actually cost
Almost every article on this costs you money cites a 100x defect-cost figure from a study that does not exist. Here is what the evidence really supports, and how to reason about visual defects without it.
If you have read anything about why testing matters, you have seen this: a bug costs $1 to fix in requirements, $10 in development, $100 in QA, and $10,000 in production. It is usually attributed to the IBM Systems Sciences Institute, and it is the load-bearing statistic of an entire genre of content marketing — including, until this article, ours.
It does not survive contact with a citation check. That is worth knowing before you put it in a slide, because someone in the room may have done the check.
The study nobody can produce
Trace the citation and it goes in a circle. Blog posts cite other blog posts. Those cite vendor whitepapers. The whitepapers cite Roger Pressman's textbook Software Engineering: A Practitioner's Approach, which attributes the figures to internal IBM course notes catalogued as "IBM81".
The "IBM Systems Sciences Institute" was an internal employee-training programme, not a research body. No underlying dataset has ever been published. The numbers, if they were ever measured at all, describe software development as it was practised in or before 1981 — mainframe era, waterfall lifecycle, punch-card-adjacent release cadence.
The related figure from Barry Boehm's Software Engineering Economics (1981) is on firmer ground — Boehm assembled real cost data at TRW with corroborating figures from IBM, GTE and the Bell Labs Safeguard programme. But it is still 1981 data about 1981 projects, and it is routinely misquoted as the IBM figure by people who have read neither.
What the evidence does support
Discard the multiplier and something useful remains. The shape holds even where the slope does not: a defect found later costs more than the same defect found earlier. That is not really an empirical claim so much as an accounting one, and you can derive it from your own process without citing anyone.
| Found at | Who is involved | What it costs you |
|---|---|---|
| Before commit | One developer, in context | Minutes. No coordination. |
| In code review | Two developers | A review cycle. Author still has context. |
| In CI | One developer, context partly lost | A re-run, a fix, another review. |
| In staging or QA | Developer, reviewer, tester | A bug report, triage, scheduling, re-test. |
| In production | Support, on-call, engineering, sometimes comms | All of the above, plus the customers who saw it. |
The escalation is not exponential in any measured sense. It is combinatorial in people: each stage adds humans, and each added human adds a handoff, a context reconstruction, and a scheduling delay. You can price that yourself with your own salary costs and your own defect-tracking data, which makes it a far stronger argument than a borrowed number.
Why visual defects escalate further than most
This is the part specific to interfaces, and it is why visual bugs deserve separate treatment rather than being folded into a general defect argument.
- Nothing alerts on them. A 500 pages your error tracker. A button rendered white-on-white throws nothing, logs nothing, and pages nobody. It is found by a customer, or it is not found.
- They survive the full test suite. Unit, integration and end-to-end tests all assert on behaviour and the DOM. A visible-to-nobody submit button satisfies every one of them — see common visual bugs.
- Time-to-detection is measured in days, not minutes. Most defect economics assumes something breaks loudly. Visual defects have no such signal, so the "found in production" column is where nearly all of them land by default.
- They are disproportionately in the highest-value paths. Shared components — buttons, forms, modals, navigation — are what break, and those live on the pages that carry revenue.
So the honest framing is not "visual bugs cost 100x". It is that visual bugs skip the cheap detection stages entirely and arrive at the most expensive one by default, because nothing in a normal pipeline is looking for them.
Numbers you can actually defend
Rather than importing a statistic, derive four numbers from your own systems. Every one is available to you and none requires a citation.
- How many visual defects reached production last year? Search your issue tracker for the ones labelled UI, layout, styling, or reported with a screenshot. Most teams are surprised by the count.
- How were they found? Sort into: caught internally, reported by a customer, found by chance. The customer-reported share is the number that gets attention in a leadership conversation.
- How long did each live before anyone noticed? Commit date to report date. Multiply by traffic on the affected route to get an order-of-magnitude impression count.
- What did each cost to resolve? Not just developer hours — include support tickets, triage, and the release process it required.
How to turn those four numbers into a spend decision is worked through in the business case for visual testing. If the pages you care about are landing and pricing pages rather than product screens, visual testing for marketing teams covers that angle.
Why we are telling you the statistic is fake
We sell visual testing. The 100x figure is extremely convenient for us, and we could have used it — nearly every competitor does.
But a business case built on a number that dissolves under a citation check is a liability for whoever presents it internally. If you take the 100x figure into a room and one person has read Bossavit, you have lost the argument and some credibility with it. The argument from your own defect data is harder to assemble and impossible to dismiss.
The case for visual testing does not need a fabricated multiplier. It needs one observation: the cheapest stages of your pipeline are structurally incapable of catching visual defects, so they all arrive at the most expensive one. Fixing that is what the category is for.
Frequently asked questions
Is it true that bugs cost 100x more to fix in production?
The specific 1x/10x/100x figure attributed to the IBM Systems Sciences Institute has no published dataset behind it. The citation traces to internal IBM course notes from 1981 referenced in Pressman's textbook, and the Institute was a training programme rather than a research body. The general principle that later defects cost more is sound; the precise multiplier is not a fact you should quote.
Where does the defect cost curve come from?
Barry Boehm published a relative cost-to-fix curve in Software Engineering Economics in 1981, based on data gathered at TRW with corroborating figures from IBM, GTE and Bell Labs. The much-repeated IBM Systems Sciences Institute version is a separate and less-supported claim. Laurent Bossavit examines both in Chapter 10 of The Leprechauns of Software Engineering.
How do you calculate what a visual bug costs?
Use your own data rather than published multipliers: count the visual defects that reached production last year, what proportion were reported by customers rather than caught internally, how long each lived before anyone noticed, and the total resolution cost including support and release overhead. Impressions on the broken route before detection is usually the most persuasive figure.
Why are visual bugs harder to catch than other bugs?
Because nothing alerts on them. A server error appears in your error tracker; a button rendered invisible throws nothing and logs nothing. Unit, integration and end-to-end tests all assert on behaviour and the DOM, so they pass regardless of appearance — meaning visual defects skip every cheap detection stage and arrive in production by default.