Are flaky tests false positives?

A flaky test that fails intermittently on working code is producing false positives. Not every false positive is flaky, though — a false positive can also come from a stale assertion, a bad selector, or an over-strict check that fails on a legitimate change. Flakiness is one common source, not a synonym.

False Positive vs. False Negative in Software Testing

Q: Is a false positive when the test fails or when it passes?

In the standard QA convention, a false positive is a test that FAILS while the feature actually works — a false alarm. This trips people up because in medicine a 'positive' result means the condition is present. In testing, the 'positive' refers to the test raising an alarm, not to the bug being real. If a term is ambiguous on your team, say 'the test failed but there was no real bug' instead.

Q: Which is worse, a false positive or a false negative?

A false negative is worse per occurrence because a real defect reaches users undetected — the test told you everything was fine. But false positives are usually more frequent, and their cumulative cost is that people stop trusting the suite and start ignoring or blindly re-running red builds, which then lets real failures hide in the noise. Track both; don't optimize one to zero at the expense of the other.

In software testing, a false positive is a test that reports a failure when the code is actually working, and a false negative is a test that passes when the code is actually broken. Put simply: a false positive is a false alarm, and a false negative is a missed bug. Both undermine a test suite, but in opposite ways — false positives waste time and erode trust, while false negatives let real defects reach users unnoticed.

What is a false positive in software testing?

A false positive is a failing test result that does not correspond to a real defect. The feature works as intended, but the test says it doesn't. The test is wrong, not the product.

Common shapes of a false positive:

A test fails because a button's CSS selector changed, even though the button still works.
An assertion checks an exact string ("Welcome back, Sam!") that legitimately changed to "Welcome, Sam" — the change was intended, but the test wasn't updated.
A flaky test fails once out of ten runs because of a timing race, not a bug.
A test depends on shared state left behind by a previous test, so it fails out of order.

The danger of false positives is cumulative. One red build that turns out to be nothing is an annoyance. A suite that is red half the time trains the team to ignore red — and then a genuine failure hides in plain sight.

What is a false negative in software testing?

A false negative is a passing test result that hides a real defect. The product is broken, but the test reports success — so nobody looks. This is the more dangerous of the two per occurrence, because the failure is silent: the safety net you were relying on told you everything was fine.

Common causes of false negatives:

The test only checks the happy path and never exercises the input that breaks.
An assertion is too weak — it checks that a request returned a 200 status but not that the response body was correct.
A test that should have run was skipped, or silently swallowed an exception.
The test asserts on implementation details that still pass while the user-facing behavior is broken.

False positive vs. false negative: why the words confuse everyone

The terminology causes real arguments in QA, so it's worth pinning down. In medicine, a "positive" result means the condition is present. In testing, "positive" refers to the test raising an alarm — not to the bug being real. So:

False positive = the test fired an alarm (failed), but there was no real problem.
False negative = the test stayed quiet (passed), but there was a real problem.

If your team keeps mixing these up, drop the jargon and use plain descriptions: "failed but no real bug" and "passed but a real bug shipped." Unambiguous words beat correct jargon nobody agrees on.

Which is worse, a false positive or a false negative?

It depends on what you're optimizing. A false negative is worse in a single instance because a defect reaches production undetected — the cost lands on users and on whoever has to debug it later without a failing test to point the way. A false positive is usually less severe individually but far more frequent, and its real cost is trust: once people expect the suite to lie, they stop reading it.

The two also trade off against each other. Make a test stricter to catch more real bugs (fewer false negatives) and you risk more false alarms on legitimate changes (more false positives). Loosen it to stop the noise and you risk missing real defects. The goal isn't zero of either — it's keeping both low enough that the suite stays trustworthy and protective at the same time.

How to reduce false positives

Treat every false positive as a bug in the test. Don't re-run until green. A test that fails on working code is broken and needs a fix, same as product code.
Quarantine, then root-cause. Move a repeatedly-flaky test out of the blocking suite so it stops training people to ignore red — but track it and fix it, don't leave it quarantined forever.
Use resilient selectors. Target stable attributes like data-testid instead of brittle CSS paths or text that copywriters change. This kills a whole class of UI false positives.
Assert on behavior, not exact incidentals. Check that the user is logged in, not that the greeting string is byte-for-byte identical.
Isolate test state. Each test should set up and tear down its own data so it doesn't fail because of what ran before it.

Selector drift is one of the biggest sources of false positives in end-to-end tests. Self-healing tests address it directly: when the UI changes, the test re-locates the element instead of failing outright. That's the idea behind Klavity AutoSim — keeping a green build honest by removing false alarms that come from harmless UI changes rather than real regressions.

How to reduce false negatives

Test the edges, not just the happy path. The bugs that escape live in empty inputs, huge inputs, error responses, slow networks, and unusual sequences — exactly the paths a passing happy-path test never touches.
Strengthen weak assertions. Don't stop at a status code; assert on the actual output that matters to the user. A test that can't fail isn't protecting anything.
Turn every escaped defect into a regression test. Each bug that reached users is proof of a false negative. Write a test that fails on the old code and passes on the fix so the same gap can't reopen. (See reducing escaped defects.)
Verify tests can actually fail. Deliberately break the code and confirm the test goes red. A test that stays green when you sabotage the feature is a permanent false negative.
Explore beyond your written cases. Scripted tests only catch what you thought to check. Structured exploration — including AI personas that navigate the product like unpredictable real users — surfaces the paths your test plan never imagined.

How to catch both in one workflow

The through-line is that both failure modes hide when your evidence is thin. A false positive is hard to dismiss without seeing what the test actually saw; a false negative is hard to prevent when the bug report that could have become a test never carried enough detail. Capturing complete state at the moment something looks wrong — the screenshot, console errors, network requests, and environment — makes it fast to tell a real failure from a false alarm and easy to convert a real one into a durable test.

That's the job of a good in-app bug report: it gives triage the evidence to sort false positives from real regressions in seconds, and hands developers exactly what they need to reproduce, fix, and lock the fix in with a test. Fewer false alarms and fewer missed bugs come from the same discipline — better evidence, faster.