Cart A/B testing lets you run two versions of your cart against each other so you can see which one actually performs better, instead of guessing. It's available on the Plus and Elevate plans.
🧠 Pro Tip: If you're testing something more specific than revenue or conversion - like which free gift customers prefer - use Manual Mode so you can dig into the advanced analytics yourself instead of relying on an automatic winner.
🛠️ How to Set It Up
Go to your cart settings page and choose Run A-B Test.
This opens the experiment creation form. Set your deciding metric — the options are Revenue Per Session, Conversion Rate, Average Order Value, or Manual Mode (use Manual Mode if you want to review the advanced analytics yourself rather than have a winner picked automatically).
Set your Variation Split — this defaults to 50-50, but you can change it if you want an uneven split.
Choose an end date (and end time) for your test.
Decide whether to automatically promote the winner when the end date is reached. If turned on, the winning cart becomes your main cart and the other variation is discarded.
If you turn on automatic promotion, set the minimum requirement per variant to make sure results are statistically accurate before a winner is picked. If you'd rather review the results yourself before promoting a winner, leave this off.
Click Save to create the test.
Your test is now created in draft mode. From here you can review and edit your original cart, or edit the new variation B that's been created for you.
To edit Variation B, click into it and make your changes — it starts out identical to your original cart. Click Save when you're done.
From the top of the page, you can navigate back to the main A-B test view, or switch to Variation A.
Once you're ready to launch, click Activate Experiment.
Once the experiment is active, your two variations start being served to different sets of customers, and you can no longer edit either cart — this keeps the test accurate. If you need to make changes while a test is running, end the test early, make your edits, then restart the A-B test.
While the Test Is Running
As data comes in, you'll see your deciding metric analysis update, along with a winning analysis showing which cart is currently ahead. Scroll down to see a full performance comparison between Variation A and B.
While a test is active, most settings are locked — the only things you can still change are:
The end date (move it earlier or later)
The variation split
Choosing a winner early
To view analytics for a variation, click View Analytics on that variation. This gives you access to all the detailed analytics data points, including things like free gift performance and uptake.
While an A-B test is running, you'll see a badge indicating the test is active, and the majority of the cart will be locked down and can't be edited while the test is running — with the exception of targeting and scheduling.
Each customer is locked to the variation they're first served for that device until they complete a purchase or the experiment ends.
To end a test early, go into the Edit A-B Testing page, scroll down, and choose the winning variation to end the test with.