> ## Documentation Index
> Fetch the complete documentation index at: https://docs.depict.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# A/B testing

> How A/B tests work: what varies between groups, how traffic is split, and how to read the results.

An A/B test runs on a single collection and measures the effect of that
collection's visual merchandising — content cards, product duplicates and
per-collection image settings. Shoppers in the test group see the collection
as merchandised; shoppers in the control group see it without those elements.

## Limits and behavior

* Tests run on one collection at a time.
* The traffic split is 50/50 and cannot be changed.
* Group assignment is per browser and persists for the whole test.
* The product order is identical in both groups: Depict writes the order into
  the Shopify collection itself, so ordering is not part of what a test
  varies. Only content cards, product duplicates and image settings differ.
* Add-to-carts and purchases only count products that are in the collection.
* A/B testing is available on plans with Merchandising at Large or Custom
  (including the earlier Pro and Custom plans), and during the Unlimited
  Trial. See [Your plan](/shopify-lite/settings#your-plan).

## Before you start

* The store's plan must include Merchandising at Large or Custom, or the
  store must be in the Unlimited Trial.
* The collection must contain at least one content card, product duplicate or
  image setting — without them, the two groups would be identical.

## Starting a test

<Steps>
  <Step title="Open the collection">
    Open the collection in the editor from **Collections**.
  </Step>

  <Step title="Start the A/B test setup">
    Start the A/B test setup from the collection editor.
  </Step>

  <Step title="Complete the Todos step">
    The **Todos** step lists anything the collection still needs before a
    test can run.
  </Step>

  <Step title="Configure the test">
    In **Settings**, enter a name, a start date, an end date, and optionally
    an email address to notify about the test.
  </Step>

  <Step title="Launch">
    In **Review**, click **Launch**.
  </Step>
</Steps>

**Result:** the collection's test page opens, with an **Insights** view, a
**Preview** view, and a **Stop** button to end the test early. Traffic starts
splitting on the test's start date.

## How the split works

Every visitor is randomly assigned to one of the two groups, 50/50. The
assignment is stored in the visitor's browser and persists for the whole
test, so a returning shopper stays in the same group on the same device. The
same person on a different device or browser can land in the other group.

## Reading the results

The Insights view shows, per group and per day:

| Metric | Definition |
| - | - |
| Page views | Views of the collection page |
| Clicks | Product clicks on the collection page |
| Add-to-carts | Unique add-to-carts of products in the collection |
| Purchases | Orders of products in the collection |
| Click-through rate | Clicks ÷ page views |
| Add-to-cart rate | Add-to-carts ÷ page views |
| Purchase rate | Purchases ÷ page views |

Statistical significance is computed with a two-tailed z-test for each rate
metric, over the whole test window so far. The p-value shows as a dash until
both groups have page views, and it is recomputed as data accumulates, so it
moves — a single early dip below a threshold is not a conclusion. Low-traffic
collections need longer test windows before the p-values are meaningful.

When the test concludes, an AI-written summary states which group won, with
what confidence, and suggests next steps.

## Troubleshooting

* **The A/B test option asks me to upgrade.** A/B testing requires the Pro or
  Custom plan; it is also available while the free trial is active.
* **The setup won't launch.** The collection must contain at least one
  content card, product duplicate or image setting.
* **Both groups look the same.** The control group hides content cards,
  duplicates and image settings; the product order is the same for everyone.
  Check which group your own browser was assigned to.
* **No p-value yet.** The p-value appears once both groups have page views.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.