Shopify CRO audit
Fixes ranked by impact, with a test plan where traffic allows.
Insight
A/B Testing With Low Traffic
Most small and mid-sized stores do not have the traffic to A/B test button colours. That does not mean they cannot improve conversion rate with evidence.
Short answer
With low traffic, a classic A/B test on a small change will rarely reach a trustworthy result in a reasonable time, because the number of conversions — not visitors — decides how quickly a test concludes. Instead: fix obvious problems without testing, test bigger changes that could move conversion substantially, measure upstream metrics such as add-to-cart that happen more often, run tests longer on your highest-traffic templates, and rely more on qualitative research.
Why low traffic breaks A/B tests
A test needs enough conversions in each variant to tell a real difference from noise. Detecting a small improvement requires far more conversions than detecting a large one. A store with a few hundred orders a month can detect a big change in a reasonable time; it cannot detect a 3% lift on a product page before the season, the traffic mix and the promotions have all changed underneath the test.
Stopping a test early because one variant "looks like it is winning" makes it worse: early leads are often noise, and peeking repeatedly at an unfinished test inflates false positives.
What to do instead
1. Fix, don't test, the obvious
Hidden delivery dates, broken mobile layouts, missing payment methods, slow pages, unclear sizing — these are not hypotheses. Fix them, then compare before and after over comparable periods.
2. Test bigger changes
A redesigned product page layout, a new offer structure, a different checkout flow — changes large enough that, if they work, the effect is big enough to detect with your traffic.
3. Measure an upstream metric
Add-to-cart and checkout-started happen several times more often than purchases. Testing a product-page change against add-to-cart reaches a result faster. Validate occasionally that add-to-cart movements carry through to purchases.
4. Test where the traffic is
Run tests on templates, not single pages: all product pages, the cart, the checkout. A change to the product template collects data from every product.
5. Use qualitative research properly
- Session recordings and heatmaps — Microsoft Clarity is free.
- Customer questions from email, chat and WhatsApp before purchase.
- Post-purchase surveys — "What nearly stopped you buying today?"
- Five-second and usability tests with people who match your customers.
Qualitative research does not prove a change works. It tells you which changes are worth making.
A simple rule of thumb
| Monthly orders | Sensible approach |
|---|---|
| Very low | Research and fix; compare before/after |
| Moderate | Test big changes on templates; use add-to-cart as the metric |
| High | Classic A/B testing with a pre-set sample size and duration |
Use a sample-size calculator before every test, decide the duration in advance, and do not stop early. If the calculator says the test will take six months, change the test, not the rules.
Where this fits
This is the approach I use in a Shopify CRO audit: the audit separates the "just fix it" findings from the genuine hypotheses, and only the second group goes into a test plan.
Frequently asked questions
How much traffic do I need for A/B testing?
It depends on your conversion rate and the size of the effect you want to detect. Use a sample-size calculator before starting; if it says the test needs many months, test a bigger change or a higher-frequency metric.
Can I A/B test on Shopify without an app?
Shopify does not include native A/B testing for most stores, so testing usually needs an app or a third-party tool. For low-traffic stores, research-led fixes often beat buying one.
Is it OK to stop a test early when one version is clearly winning?
Usually not. Early leads are often noise. Decide the sample size and duration up front.
What should a small store test first?
Product page layout and information, shipping-cost clarity, and checkout friction — the changes most likely to make a large difference.
Related
Conversion Rate Optimisation
Turning the traffic you already pay for into customers.
Shopify store getting traffic but no sales
The order to check things in.
Glossary
Plain-English definitions of CRO and paid media terms.
Let's Talk
Let Me Help You Grow Your Business.
Book your free performance analysis call. No pressure, no obligations, just honest insights about your marketing performance.
Let’s Talk