Articles on: Offers

A/B tests

An A/B test shows your shoppers two to four versions of one offer and measures which one earns more. Try a bigger discount against a smaller one, a different set of bundle options, or new titles and badges, then keep the version that wins.


To find A/B tests, go to Supr Bundles & Subscriptions > Grow and open A/B tests.


Table of contents



How an A/B test works


A test starts from one of your offers. That offer, as it is now, is variant A. Each other variant (B, C and D) is a copy of it that you change in Offers, so a variant can have different discounts, subscription options, bundle options, gifts, add-ons, texts and badges.


Once the test runs, each shopper is shown one variant and keeps seeing that same variant on every visit from the same browser. Each variant prices its own options at checkout, exactly like any other offer. The app counts how many shoppers saw each variant, how many added it to their cart, and what they bought, and tells you which variant earns the most per visitor.


These stay the same in every variant, because they decide who sees the offer rather than what it offers:


  • the products the offer is on,
  • Markets, Customers and Schedule,
  • the design the offer draws with.


They follow variant A: change them on variant A and save, and every other variant follows.


Creating a test


  1. Go to Grow > A/B tests and click Create A/B test.
  2. Under Offer to test, choose the offer. Only published offers that are not already in a test are listed.
  3. Give the test a name, or keep the one suggested.
  4. Under Variants, choose 2 (A and B), 3 (A to C) or 4 (A to D).
  5. Under Traffic split, choose Even to give every variant the same share of shoppers, or Custom to choose the share that sees the other variants (from 5% to 95%). With Custom, variant A gets the rest, and the other variants share their part equally. The line under it shows the split, for example A 70%, B 15%, C 15%.
  6. Click Create test.


The test is saved as a draft. Its variants B and on are created in Offers as drafts with the same name as the offer, because the name is the title shoppers see. In Offers, each variant has an A/B test badge with its letter, for example A/B test: B. Nothing changes on your store until you start the test.


Editing the variants


On the test's page, click Edit next to a variant to open it in Offers. A line at the top says which test and variant it belongs to.


While a test is a draft or running, the Preview in each variant's editor has buttons for every variant: A, B and so on. Click one to open that variant, with its own preview, so you can move between the variants and compare them as you edit. Save your changes first: with unsaved changes, the editor stays where it is until you save or discard them. A/B test next to them opens the test's page.


Change what you want to test, then save. For a clear result, change one thing per variant, for example only the discount, so you know what made the difference.


On variants B and on, the products are greyed out, Markets, Customers and Schedule are not shown, and the design reads Design follows variant A. Change them on variant A.


Starting the test and previewing each variant


When every variant is ready, click Start test on the test's page. Every variant is published and shoppers start being split between them. While the test runs, a variant cannot be set to draft or deleted in Offers.


Once it runs, click Preview next to a variant to open the product page showing that variant, in a new tab. The preview applies to your own browser only, so you can check each variant before shoppers buy it. Place a test order in each variant if you want to check checkout too.


You can rename the test or change the traffic split at any time while it runs. Under Settings, make the change and click Save. Changing the split moves some shoppers who already saw the offer to another variant, so settle on it before you start if you can.


Reading the results


The Results on the test's page cover the whole test by default. Use the date button to look at part of it.


The line at the top says how the test stands:


  • When a variant clearly earns more per visitor, it names that variant, how much more it earns than the next best, and says you can be 95% confident the difference comes from your change and not from chance. That variant gets a Significant winner badge.
  • Until then, it says to keep the test running, and roughly how many more visitors each variant needs before the result is reliable.


Revenue per visitor decides the winner. It weighs both how many shoppers buy and how much they spend, so a bigger discount that sells more but earns less does not win by accident.


The cards, the chart (one line per variant, pick the metric above it) and the variants table use these numbers:


  • Visitors: shoppers who saw the variant, each counted once a day.
  • Views: times a page showed the variant, reloads included.
  • Add to cart and Add to cart rate: times shoppers added the variant to their cart, and that out of its visitors.
  • Orders and Conversion rate: orders with a product bought through the variant, and that out of its visitors.
  • Revenue: what the products bought through the variant sold for, after discounts.
  • Added revenue: what bundles sold for beyond one unit at full price. A $18 bundle of a $10 product adds $8.
  • Average order value: revenue divided by orders.
  • Revenue per visitor: revenue divided by visitors.
  • Bundle orders, Subscribed orders and Subscription rate: orders with a bundle, orders with a subscription, and the share of orders with a subscription.
  • Units per order: units bought through the variant divided by orders.
  • Chance to beat A: for variants B and on, how likely it is that the variant really earns more per visitor than variant A.


The Options table under it breaks each variant down by option: how often each subscription option, bundle option and the one-time purchase was chosen, added to the cart, ordered and what it earned.


A subscription's first order counts toward its variant. Later renewals do not, so a variant that sells more subscriptions can be worth more over time than its numbers here show.


Ending the test and keeping the winner


Click End test. The window lists every variant with its visitors, conversion rate and revenue per visitor, and picks the significant winner if there is one. Choose the variant to keep and click End and keep with its letter.


  • The winning variant stays published and is shown to every shopper.
  • The other variants go back to drafts in Offers, where you can delete them or reuse them later.
  • The results stay on the test's page, and each variant's numbers stay in Analytics > Offers, where each variant is an offer with its own performance page.
  • Click View above the variants on the test's page to see every variant's widget side by side, as it was when the test ended, even if you later edit or delete a losing variant. Before the test ends, View shows the variants as they are now.


A test does not end on its own and never picks a winner for you.


To delete a test, open it and click Delete. Deleting a draft test also deletes its draft variants B and on; variant A's offer stays as it is. Deleting an ended test keeps every variant's offer. End a running test before you delete it.


What is not measured yet


  • Checkout rate: how many shoppers reached checkout.
  • Profit: revenue minus your product costs.
  • Refunds and cancelled orders: an order counts when it is placed, even if it is refunded later.


Common questions


How long should I run a test?
Until the results line names a significant winner. With little traffic that can take weeks. Ending earlier is your call, but the numbers may still be down to chance.


Will a shopper see a different variant tomorrow?
No. A shopper keeps their variant on every visit from the same browser. On another device or after clearing their browser data, they may get another one.


Can I test two different offers against each other?
A test always starts from one offer and its copies. To compare two existing offers, create a test from one of them and set up variant B like the other.


Can I test widget designs?
Not yet. Every variant draws with variant A's design.


Does an A/B test change how checkout prices work?
No. Each variant prices its own subscription and bundle options, like any offer. A shopper only ever sees one variant, so they pay that variant's price.



Updated on: 10/13/2026

Was this article helpful?

Share your feedback

Cancel

Thank you!