Stop Criteria: A Checklist for Deciding to Continue, Change or Stop a Test

Lesson 72 of 118 · Account Academy

  • Sections8

Write the decision rules before launch

A test is easier to manage when its end conditions are written down before traffic starts. Without them, each new result can become a reason to keep going, change direction or stop early. That makes it difficult to tell whether a decision came from the plan or from the pressure of the moment.

Use a short written checklist for each test. It should state the spending limit, the evidence needed before making a judgment, the time limit, the conditions that require a pause, and the narrow circumstances in which one variable can change. It should also say what happens when a limit is reached: continue as planned, make one permitted adjustment, or stop.

Keep the rules tied to the test itself. A campaign on Meta Ads, Google Ads or TikTok Ads may have different setup details, but the basic decision process is the same. Record the platform, the account and campaign involved, the offer or destination being tested, and the version of the plan being evaluated. This makes it clearer which rules apply and helps prevent unrelated changes from blurring the result.

A useful rule is specific enough that someone reviewing the plan later could tell whether it was followed. “Stop if performance is bad” leaves too much room for interpretation. A better rule defines the spending or time boundary and the decision that follows it, without relying on a feeling that the numbers look disappointing.

Set a spending limit as a planning boundary

Before launch, decide the maximum amount of test budget that can be used without requiring a new review. Treat this as a planning scenario, not a forecast of what the test will earn or how much data it will produce. The limit is a boundary on exposure, not a promise that the test will reach a conclusion.

Write down what the limit applies to: a single campaign, a test as a whole, or a particular creative or audience. Also note whether the limit includes only media spend or other costs that are part of the test. Avoid adding money simply because the initial plan feels too small after results arrive. If more budget is appropriate, make that a separate decision with a new written rationale and an updated plan.

A spending rule should say what happens when the boundary is reached. For example, the test may stop automatically, or it may pause for a review before any additional spend is approved. The important point is that reaching the limit triggers the same process regardless of whether the latest result looks encouraging or discouraging.

If a platform account has a billing issue, unexpected activity or an access problem, do not treat the remaining budget as a reason to keep the campaign running. The account is a working tool, and protecting its integrity is part of the test plan. Resolve the issue through the platform’s normal processes before considering a restart.

Define the evidence threshold before judging

A test should not be judged merely because it has been live for a while or because a few early events look promising. Decide in advance what minimum amount of usable evidence is needed for a decision. The threshold might be expressed in terms of delivery, eligible traffic, completed actions, or another test-specific condition. Choose a measure that is relevant to the question being tested.

The threshold is not a claim that the result will be statistically conclusive or commercially successful. It is a guardrail against changing the plan before there is enough information to make even a practical decision. If the test cannot reach the threshold within its budget or time boundary, that is a planning outcome to record—not a reason to silently redefine what counts as enough.

Write down how to handle incomplete or unreliable evidence. If tracking is broken, events are not being recorded as intended, or the destination is unavailable, the test may not be producing usable evidence. In that case, pause to investigate rather than treating missing information as proof for or against the idea. Keep the original threshold visible and document why the evidence cannot be evaluated.

The checklist should also distinguish between reaching the threshold and deciding what the result means. This article is about the decision rules, not a universal way to interpret performance. The threshold simply marks when the planned review can happen. It does not dictate a particular outcome.

Put a time limit on the test

A test needs a time boundary as well as a spending boundary. Otherwise, low or inconsistent delivery can leave a campaign running indefinitely while the team waits for a clearer answer. Choose a defined period that fits the test and write down when the review will take place. The exact period depends on the plan; it should not be changed after launch just because the result is uncomfortable.

State what happens if the test reaches its time limit before it reaches the evidence threshold. Possible decisions include stopping, pausing for a technical review, or treating the test as inconclusive and designing a new one. Choose the response before launch. Do not extend the test automatically because it is “almost there” unless the written rules already allow a specific review and extension process.

The time boundary also helps separate a test from an ongoing campaign. If the test ends, stop or pause the relevant activity as planned. Continuing to spend while postponing the decision turns the limit into a suggestion. A later, separate plan can use what was learned, but it should not be presented as though the original test followed its rules.

Define pause conditions for account and policy trouble

Some conditions require a pause regardless of performance. Write them down before launch so that safety and compliance are not weighed against the desire to collect more data. Examples include a platform notice that needs attention, a restriction or review affecting the account, a billing or access problem, a destination that no longer works as intended, or a concern that the campaign may not comply with the platform’s terms.

A pause is not the same as abandoning the test. It is a stop in activity while the issue is understood through legitimate platform channels and the relevant setup is checked. Do not move the campaign to another account, alter identities, or use workarounds to evade a platform review. If an account was purchased, treat it as a product whose ownership, access and permitted use need to be clear; a purchase does not remove the need to follow platform requirements.

Write down who is responsible for checking the issue and what must be true before activity can resume. If the problem cannot be resolved appropriately, the decision may be to stop. Do not let a spending target, deadline or promising early signal override a pause condition.

Allow one controlled change, not a moving plan

A test may need a change when a setup problem prevents it from answering the intended question. Decide in advance which kind of correction is allowed and what would trigger it. For example, the plan might permit changing one variable after a specified review, while keeping the audience, offer, placement or other chosen elements fixed. The particular variable depends on the test; the rule is to change one thing at a time.

Record the original version and the reason for the change. Then treat the adjusted setup as a new version with its own review point, budget boundary and evidence threshold. Do not combine several edits and call the result a continuation of the same test. When many elements change together, it becomes difficult to know which decision the evidence relates to.

A single-variable change is not permission to keep searching for a version that produces a preferred answer. It is a controlled response to a defined condition. If the test reaches its limit and the allowed change does not address the original question, stop and plan a separate test rather than adding further adjustments on the fly.

Use the checklist when emotions push back

Before launch, ask whether the written rules still seem reasonable when imagining both an encouraging result and a disappointing one. If the plan says to stop at the spending boundary, will that still feel acceptable after a promising day? If the evidence threshold has not been reached, will there be a temptation to declare a winner anyway? If the result is weak, will it be tempting to extend the time limit because stopping feels like admitting failure?

At review, read the plan before looking for a reason to continue or stop. Mark each condition as met, unmet or not assessable, and record the evidence behind that classification. If a rule is unclear, do not resolve the ambiguity in whichever direction is more convenient. Pause the decision, document the uncertainty, and make a clearer rule for the next test.

Emotions are not evidence that the plan should be ignored. They can signal that the plan was incomplete, that the budget feels uncomfortable, or that a result has become personally important. Note those reactions separately from the decision record. If circumstances have genuinely changed, a new plan may be justified—but it should be described as a new decision, not as proof that the original rules were followed.

Keep the line where it was drawn

Moving a limit after seeing the result undermines the purpose of having stop criteria. If the spending boundary changes only when performance is disappointing, the test can continue until it produces a preferred answer. If the evidence threshold changes only when a result looks strong, the same problem appears in the opposite direction. Either way, the decision is no longer being made under the conditions set before the outcome was known.

That does not mean every plan must be followed blindly. A platform issue, a material change in the offer, or a defect in the setup can make the original test invalid. In those cases, stop or pause, explain what changed, and decide whether a new test is warranted. Keep the original record intact so that the reason for the change remains clear.

Before launch, the final checklist should answer five questions: What is the maximum planned spend? What evidence threshold must be met before review? When does the test end? What account, access or policy conditions require a pause? What single variable, if any, may change, and under what condition? Once those answers are written down, follow them consistently. That is what makes a decision to continue, change or stop a deliberate part of the test rather than a reaction to its latest result.