Why self-authored tests validate the wrong thing
A test suite written by the same agent that wrote the implementation mostly proves internal consistency, not correctness against the actual requirement. High coverage in this situation tells you the code does what the agent thought it should do — which is precisely the thing in question.
Independent test authorship, ideally from the original requirement rather than from reading the implementation, is what actually closes this gap.
What to measure instead of, or alongside, coverage
Mutation testing — deliberately introducing bugs and checking whether the test suite catches them — reveals whether tests are actually asserting behaviour or just executing lines. Pairing it with requirement-derived acceptance tests, written independently of the implementation, gives a far more honest signal than coverage percentage alone.
A green pipeline around a shared misunderstanding is still a green pipeline. Coverage can't tell the difference.MNB Software Engineering Practice
Key actions
- Write acceptance tests from the requirement, independent of the implementation
- Use mutation testing to check tests actually assert behaviour
- Treat high coverage with a single-author agent as a flag, not reassurance
- Review tests for what they assume, not just whether they pass