The Coverage We Forgot

AI-accelerated delivery needs a broader definition of coverage before teams can release with confidence.

The Missing Question

AI-assisted software delivery is forcing us to revisit an old question with new urgency.

The old question was: Did we write enough tests?

The better question is: Did we cover enough reality?

Code coverage asks whether tests executed parts of the code. That matters, but the real world does not only attack code paths. It attacks assumptions.

A system can have high code coverage and still fail because it was never tested against the right signals, data conditions, lifecycle states, timing patterns, accumulated events, business calendars, aggregate relationships, human behaviors, or downstream consequences.

That is the coverage we forgot.

If AI helps teams build faster, then validation has to move earlier and become more explicit. We cannot let generated implementation outrun the evidence needed to trust it.

The practical move is simple:

Do not just build the solution. Build how the solution will prove itself.

The Trap: Generated Tests Can Encode the Same Misunderstanding

Generated tests are useful. They can save time, catch mistakes, and help teams move faster.

But generated tests often encode the same misunderstanding as the generated code, the requirement, the user story, the acceptance criteria, the business analyst’s interpretation, the tester’s assumption, or the developer’s prompt.

GenAI has accelerated this preexisting SDLC challenge. AI did not invent shallow requirements. AI did not invent misunderstood business rules. AI did not invent happy-path testing. AI did not invent the dangerous confidence that comes from a polished demo.

Before GenAI, many projects were tested over days, weeks, or months, depending on the system and the risk. During that time, misunderstandings had more chances to surface. A month-end process ran. A quarter-end close exposed an aggregation issue. A re-org changed ownership. A stale reference table broke a lookup. A late-arriving file hit the system. A user did something no one included in the requirements because everyone assumed “that never happens.”

Time does not find everything. But time finds many things, including the evidence needed to deploy with trust and confidence.

AI compresses that elapsed time. That can be good. It can also remove the accidental discovery window that used to expose weak assumptions before release.

So the new SDLC has to replace accidental discovery with intentional coverage.

Code Coverage Is Not Enough

We need more explicit language for the coverage we often forget.

Signal Coverage

Signal coverage asks whether testing exercises the meaningful events, messages, triggers, workflow actions, upstream feeds, user behaviors, system states, and operational signals the system must interpret.

Examples include:

A contract is submitted.
A signature packet is sent.
A customer signs late.
A file arrives twice.
A record is updated out of sequence.
An approval is revoked.
A user changes roles.
A downstream refresh runs before the upstream feed is complete.

The question is more than whether the system ran without an error. We need to know whether the system saw the signals it will actually see in the wild.

Data Condition Coverage

Data condition coverage asks whether testing exercises the meaningful shapes, states, defects, ranges, boundaries, combinations, and lifecycle positions of the data the system must process.

Examples include:

Missing effective dates.
Overlapping date ranges.
Duplicate business keys.
Deleted parents with active children.
Late-arriving facts.
Historical corrections.
Multi-currency values.
Empty states.
Old migrated records.
Active records with expired reference data.
High-value edge cases.
Rare but valid customer types.

Clean test data is useful. But production data is often a field journal of human compromise.

Temporal Coverage

Temporal coverage asks whether the system has been tested against elapsed time, waiting periods, business calendars, deadlines, retries, expiration, renewal windows, and clock-dependent rules.

A rule like “send a reminder after three days” is rarely as simple as it sounds.

Three calendar days or three business days?
What if day three falls on a holiday?
What if the contract was withdrawn?
What if the customer partially signed?
What if the owner changed during the waiting period?

Accumulation Coverage

Accumulation coverage asks whether the system has been tested against repeated events, threshold crossings, cumulative magnitude, counts over time, and rolling windows.

Examples include:

Three unanswered requests trigger escalation.
Total exposure crosses an approval threshold.
Three late deliverables in 90 days increase risk.
Repeated failed imports trigger operational review.
Millions of tiny rounding differences become material.
A system maximum is crossed only after months of ordinary usage.

Sometimes the failure comes after events accumulate.

Lifecycle Coverage

Lifecycle coverage asks whether the system has been tested across meaningful entity states and transitions.

A contract is not merely a row in a table. It is a lifecycle:

Draft. Submitted. Under review. Approved. Sent for signature. Signed. Active. Amended. Suspended. Renewal pending. Expired. Terminated. Archived.

Different rules should fire in different states. Other rules should not fire. Skipping lifecycle coverage is how systems pass tests and still misbehave in production.

Intentional Calendar Coverage

In well-run projects, calendar coverage is not accidental. It is intentional.

A team may deliberately plan test windows to cover month-end, quarter-end, year-end, fiscal close, reorganizations, downstream refreshes, reporting cycles, renewal windows, and operational handoffs.

That is not waste. That is part of the validation strategy.

Intentional calendar coverage reduces the need to manufacture every date condition, fake every clock event, or build elaborate synthetic data for every temporal edge case. For some systems, it is the difference between reaching a sufficient level of trust to deploy and walking away because the team cannot reach minimum confidence.

Sometimes the system looks good. It may even be good. But if the team did not design a mechanism to move from “this looks right” to “we can prove this is trustworthy,” deployment becomes too costly or too risky.

Example 1: Contract Signature Reminder

flowchart TD
    A[Contract approved] --> B[Signature packet sent]
    B --> C{Customer signed?}

    C -->|Yes, before 3 business days| D[Mark signature received]
    D --> E[No reminder needed]

    C -->|No| F{3 business days elapsed?}
    F -->|No| G[Wait and monitor]
    G --> C

    F -->|Yes| H{Contract still active in signature process?}
    H -->|No, withdrawn or cancelled| I[Suppress reminder]
    H -->|Yes| J[Send reminder to customer]

    J --> K{Reminder count >= 3?}
    K -->|No| G
    K -->|Yes| L[Escalate to contract owner]

    L --> M{Owner changed during waiting period?}
    M -->|Yes| N[Route escalation to current owner]
    M -->|No| O[Route escalation to original owner]

Why this example matters

A generated happy-path test may prove that a contract can move from approved to sent for signature to signed.

But the real business behavior depends on time, state, accumulated signals, and ownership context.

The system needs to know whether three business days elapsed, whether weekends and holidays count, whether the reminder clock resets when a packet is resent, whether the contract was withdrawn before the reminder fired, whether partial signature counts as a response, whether repeated reminders should escalate, and whether escalation routes to the original owner or the current owner after reassignment.

This is not just code coverage. The same code path might be executed while the actual business risk remains untested.

This is temporal coverage, lifecycle coverage, signal coverage, and accumulation coverage. The requirement sounds simple: Send a reminder after three days.

But the trust lives in the details: Three days after what? Business or calendar days? What counts as a response? What resets the clock? What suppresses the reminder? Who owns the escalation if ownership changed?

Example 2: The Rounding Error That Waited for Scale

flowchart TD
    A[Customer activity event] --> B[Calculate fractional charge]
    B --> C[Round for display]
    B --> D[Store billable amount]

    D --> E[Append to customer ledger]
    E --> F{Daily account total reconciles?}
    F -->|Yes| G[Continue processing]
    F -->|No| H[Flag account variance]

    G --> I[Aggregate across customers]
    I --> J{Monthly platform total within tolerance?}
    J -->|Yes| K[Publish billing summary]
    J -->|No| L[Hold billing summary]

    L --> M[Replay events with high precision]
    M --> N{Rounding policy mismatch found?}
    N -->|Yes| O[Correct calculation policy]
    N -->|No| P[Escalate reconciliation review]

Why this example matters

Some failures do not announce themselves in a unit test. They wait for volume.

Imagine a usage-based billing, rewards, energy, logistics, advertising, or transaction-fee system. The rule may sound harmless: Calculate the charge, round it, and add it to the account.

A generated unit test may prove that one transaction rounds correctly. An integration test may prove that one customer invoice renders correctly. A demo may look perfect.

But the system may still be wrong if one component rounds half up, another uses banker’s rounding, another stores binary floating-point values, and another reconciles from displayed amounts instead of source events.

At ten transactions, the difference is invisible. At ten thousand, it may look like noise. At ten million, the difference may become a material accounting variance, a customer trust problem, or a regulatory issue.

The same class of failure appears when an allowed maximum is never tested at realistic scale. A queue limit, retry counter, batch size, row count, precision limit, partition key, identifier sequence, or API quota can look fine in development and still fail only after ordinary usage accumulates.

This is accumulation coverage, aggregate coverage, data condition coverage, and observability coverage. The question is not only whether the calculation works once. The question is whether the system can prove its totals, replay its events, explain its rounding policy, and detect drift before customers or auditors do.

Test-Driven Vibing

What if we stop letting generation outrun validation?

Test-driven vibing (TDV) means using GenAI to accelerate solution delivery only after the team has defined the validation strategy, trust boundaries, coverage model, and evidence requirements.

We are not suggesting that we write or vibe every test before every line of code. We are imagining the human-AI team starting with a trust model:

What must be true before this system can be believed?

Only then should the team ask AI to help generate implementation, test cases, synthetic data, replay scripts, observability hooks, audit checks, and regression suites.

Test-driven vibing begins before the prompt that builds the solution. It begins with the prompt that asks what kind of evidence the system must produce before humans are allowed to trust it.

Build for Validation

When I design systems, I build in how I am going to validate them.

That means observability is not an afterthought. Reconciliation is not an afterthought. Auditability is not an afterthought. Test data strategy is not an afterthought. Replayability is not an afterthought. Failure visibility is not an afterthought.

If we want to keep up, I suggest we apply the same principle to AI-generated and AI-assisted systems.

If GenAI helps us build faster, then we need to be even more deliberate about building systems that can prove what they did, why they did it, what inputs they saw, what rules fired, what assumptions were applied, what outputs changed, and what downstream effects followed.

For some low-risk scenarios, generated tests may be enough. For higher-risk systems, generated tests are only one part of the evidence model. The team still needs to decide:

What can be mocked?
What can be simulated?
What can be replayed?
What requires production-like data?
What requires real calendar exposure?
What requires human review?
What can safely be ignored because the risk is low?
What must be observed over time before deployment?

This creates a practical delivery motion:

Define the trust goal before implementation.
Inventory the coverage classes that matter.
Map each coverage item to evidence.
Decide what AI can generate, simulate, replay, or monitor.
Decide what still requires human review.
Release only when the evidence matches the risk.

That is how AI-assisted delivery becomes something clients can trust instead of something teams merely hope will work.

Toward QA.md

Software teams are already learning how to guide coding agents with files like AGENTS.md, SKILLS.md, project constitutions, implementation plans, and repository-specific instructions.

The next step is explicit quality guidance.

Call it QA.md: a living quality contract that tells the human-AI delivery system how this project earns trust.

QA.md is not a process decoration. It is the project’s instruction file for validation. It tells the agent, the developer, the tester, the product owner, and the reviewer what evidence matters before release.

A useful QA.md might include:

# QA.md

## System Trust Goals
What must be true before this system can be trusted?

## Risk Classification
What happens if this system is wrong?

## Critical Lifecycles
Which entities have meaningful state transitions?

## Signal Coverage
Which events, triggers, messages, feeds, or human actions must be tested?

## Data Condition Coverage
Which data shapes, defects, boundaries, and historical conditions must be tested?

## Temporal Coverage
Which rules require elapsed time, business calendars, delays, retries, or waiting periods?

## Accumulation Coverage
Which rules depend on counts, thresholds, repeated events, or cumulative magnitude?

## Aggregate and Reconciliation Rules
Which totals, rollups, balances, counts, or derived values must reconcile?

## System Limits
Which maximums, quotas, precision limits, volume limits, and scale assumptions must be tested?

## Observability Requirements
What logs, traces, checkpoints, explainability hooks, and audit records are required?

## Test Data Strategy
What can be synthetic, what must be production-like, and what needs real calendar exposure?

## Replay Strategy
Can events, feeds, and state transitions be replayed?

## Human Review Points
Where must SMEs, testers, BAs, architects, security, legal, or operations review the result?

## Deployment Confidence Standard
What evidence is required before release?

The point of QA.md is not to slow the team down. Systems that get released eventually figure this out, often through incidents, customer complaints, audit findings, rework, or operational pain.

We are looking for ways to reduce the pain of discovering missed coverage after release by designing for evidence before release.

Voices We Are Echoing

We are not the first to connect AI-assisted development with test-first discipline. This article is part of a growing conversation.

ASOS has written about Test-Driven Vibe Development as a quality-first lifecycle for AI-assisted delivery. Other practitioners are writing about test-driven vibing, test-first prompting, spec-driven development with coding agents, and repository-grounded behavioral testing for coding agents.

That matters because this is not a private worry. Many teams are reaching for the same shape of answer: AI can help us go faster, but only if the delivery system becomes more explicit about intent, evidence, validation, and human judgment.

Our contribution is adjacent but different. AI-accelerated delivery needs not only test-first coding practices, but also explicit signal coverage, data condition coverage, lifecycle coverage, temporal coverage, accumulation coverage, aggregate coverage, system-limit coverage, and, when appropriate, intentional calendar coverage.

I am keeping the source trail separately in the assets vault so this article can stay readable while the research base grows.

Source collection: vaults/assets/Sources/AI-Assisted SDLC Source Index.md

The New Standard

The old question was: Did we write enough tests?

The better question is: Did we cover enough reality?

Reality is bigger than the code base. It includes signals, data conditions, lifecycle states, business cycles, elapsed time, accumulated events, aggregate values, system limits, human behavior, and downstream consequences.

If we tell GenAI what kind of coverage we need, then GenAI can help us generate the tests, simulations, replay scripts, observability hooks, audit checks, and review prompts that lead to trust.

Not just more testing artifacts.

Evidence.