Skip to main content
An A/B test runs on a single collection and measures the effect of that collection’s visual merchandising — content cards, product duplicates and per-collection image settings. Shoppers in the test group see the collection as merchandised; shoppers in the control group see it without those elements.

​
Limits and behavior

  • Tests run on one collection at a time.
  • The traffic split is 50/50 and cannot be changed.
  • Group assignment is per browser and persists for the whole test.
  • The product order is identical in both groups: Depict writes the order into the Shopify collection itself, so ordering is not part of what a test varies. Only content cards, product duplicates and image settings differ.
  • Add-to-carts and purchases only count products that are in the collection.
  • A/B testing is available on the Pro and Custom plans, and during the free trial.

​
Before you start

  • The store must be on the Pro or Custom plan, or in the free trial.
  • The collection must contain at least one content card, product duplicate or image setting — without them, the two groups would be identical.

​
Starting a test

1

Open the collection

Open the collection in the editor from Collections.
2

Start the A/B test setup

Start the A/B test setup from the collection editor.
3

Complete the Todos step

The Todos step lists anything the collection still needs before a test can run.
4

Configure the test

In Settings, enter a name, a start date, an end date, and optionally an email address to notify about the test.
5

Launch

In Review, click Launch.
Result: the collection’s test page opens, with an Insights view, a Preview view, and a Stop button to end the test early. Traffic starts splitting on the test’s start date.

​
How the split works

Every visitor is randomly assigned to one of the two groups, 50/50. The assignment is stored in the visitor’s browser and persists for the whole test, so a returning shopper stays in the same group on the same device. The same person on a different device or browser can land in the other group.

​
Reading the results

The Insights view shows, per group and per day: Statistical significance is computed with a two-tailed z-test for each rate metric, over the whole test window so far. The p-value shows as a dash until both groups have page views, and it is recomputed as data accumulates, so it moves — a single early dip below a threshold is not a conclusion. Low-traffic collections need longer test windows before the p-values are meaningful. When the test concludes, an AI-written summary states which group won, with what confidence, and suggests next steps.

​
Troubleshooting

  • The A/B test option asks me to upgrade. A/B testing requires the Pro or Custom plan; it is also available while the free trial is active.
  • The setup won’t launch. The collection must contain at least one content card, product duplicate or image setting.
  • Both groups look the same. The control group hides content cards, duplicates and image settings; the product order is the same for everyone. Check which group your own browser was assigned to.
  • No p-value yet. The p-value appears once both groups have page views.