Segment insights
The Segment insights table, at the bottom of the experiment dashboard, breaks down your experiment results by pre-defined customer segments. Use it to find the customer groups where one variant outperforms the other for a specific metric. A variant that looks inconclusive overall can still be the winning variant for a specific segment, for example, customers with a high gross average order value.
The Segment insights table appears only after the experiment has collected enough data. To learn more, see Read the Segment insights table.
Talon.One discovers these segments automatically and only shows those with high confidence. When a variant performs better for a segment on one metric, confirm this with a follow-up experiment targeting that segment, instead of discarding an inconclusive experiment.
Pre-defined segments
The pre-defined segments fall into three dimensions. A dimension is a session or customer characteristic, split into segments such as low, medium, and high order value. The dimensions are Order value, Units per order, and Customer type.
Only closed sessions with at least one effect applied are included, where each closed session represents one order. For the Customer type dimension, an order is included only if it also has a valid customer profile.
For the Order value and Units per order dimensions, Talon.One sorts orders into low, medium, and high thirds. It uses the same cut-off points for both variants, so the segments stay comparable. For Order value, it measures each order's value after any returns.
The following table lists the segments in each dimension:
| Dimension | Segment | Definition |
|---|---|---|
| Order value | Low average order value (AOV) | Orders in the bottom third by order value. |
| Order value | Medium average order value (AOV) | Orders in the middle third by order value. |
| Order value | High average order value (AOV) | Orders in the top third by order value. |
| Units per order | Low average units per order (UPO) | Orders in the bottom third by item count. |
| Units per order | Medium average units per order (UPO) | Orders in the middle third by item count. |
| Units per order | High average units per order (UPO) | Orders in the top third by item count. |
| Customer type | New customers | Customers with one completed order across the entire Application history. |
| Customer type | Returning customers | Customers with two or three completed orders across the entire Application history. |
| Customer type | Frequent customers (4+ orders) | Customers with four or more completed orders across the entire Application history. |
Read the Segment insights table
Talon.One groups segment insights into tabs, one for each metric. A tab appears only when at least one segment reaches high confidence for that metric, so fewer than three tabs can appear. The number next to each tab shows the count of segments it contains. The tabs cover the same three average metrics shown in the evaluation table:
- Gross average order value (AOV)
- Net average order value (AOV)
- Average units per order (UPO)
Within each tab, segments are sorted by confidence score, from highest to lowest. The table includes the following columns:
| Column | Description |
|---|---|
| Segment | The name of the segment and the dimension it belongs to. |
| Confidence | The confidence score for the difference between the variants, displayed as a percentage. The score is calculated using Welch's t-test. |
| Variant A | The metric value for Variant A, followed by the number of orders in the segment. |
| Variant B | The metric value for Variant B, followed by the number of orders in the segment. |
| Difference (B vs A) | The difference between the two variants, displayed as a value and as a percentage. |
| Winning variant | The variant with the best score for the segment. |
Talon.One shows a segment only when it has enough data and statistical confidence. The table can be missing segments, or appear entirely empty, for these reasons:
- Low traffic: The experiment has fewer than 100 orders per variant, or an individual segment has fewer than 100 orders per variant.
- Low confidence: A segment does not reach the 95% confidence threshold.
- Too early: The experiment has not run long enough yet. Segments can appear as it collects more orders.
A dimension is also hidden when its orders are too similar to split into separate segments.