Segment insights
The Segment Insights table, at the bottom of the experiment dashboard, breaks down your experiment results by pre-defined customer segments. Use it to find the customer groups where one variant outperforms the other for a specific metric. A variant that looks inconclusive overall can still be the winning variant for a specific segment, for example, customers with a high average session value.
Talon.One discovers these segments automatically and only shows those with high confidence. When a variant performs better for a segment on one metric, confirm this with a follow-up experiment targeting that segment, instead of discarding an inconclusive experiment.
Pre-defined segments
The pre-defined segments fall into three dimensions. A dimension is a session or customer characteristic, split into segments such as low, medium, and high session value. The dimensions are Session value, Items per session, and Customer type.
Only closed sessions are included. For the Customer type dimension, a session is included only if it also has a valid customer profile.
For the Session value and Items per session dimensions, Talon.One sorts sessions into low, medium, and high thirds. It uses the same cut-off points for both variants, so the segments stay comparable. For Session value, it measures each session's value after any returns.
The following table lists the segments in each dimension:
| Dimension | Segment | Definition |
|---|---|---|
| Session value | Low Avg. Session Value | Sessions in the bottom third by session value. |
| Session value | Medium Avg. Session Value | Sessions in the middle third by session value. |
| Session value | High Avg. Session Value | Sessions in the top third by session value. |
| Items per session | Low Avg. Items per Session | Sessions in the bottom third by item count. |
| Items per session | Medium Avg. Items per Session | Sessions in the middle third by item count. |
| Items per session | High Avg. Items per Session | Sessions in the top third by item count. |
| Customer type | New customers | Customers with one closed session across the entire Application history. |
| Customer type | Returning customers | Customers with two or three closed sessions across the entire Application history. |
| Customer type | Frequent customers (4+ sessions) | Customers with four or more closed sessions across the entire Application history. |
Read the Segment Insights table
Talon.One groups segment insights into tabs, one for each metric. A tab appears only when at least one segment reaches high confidence for that metric, so fewer than three tabs can appear. The number next to each tab shows the count of segments it contains. The tabs cover the same three average metrics shown in the evaluation table:
- Avg. Session Value
- Avg. Discounted Session Value
- Avg. Items per Session
Within each tab, segments are sorted by confidence score, from highest to lowest. The table includes the following columns:
| Column | Description |
|---|---|
| Segment | The name of the segment and the dimension it belongs to. |
| Confidence | The confidence score for the difference between the variants, displayed as a percentage. The score is calculated using Welch's t-test. |
| Variant A | The metric value for Variant A, followed by the number of sessions in the segment. |
| Variant B | The metric value for Variant B, followed by the number of sessions in the segment. |
| Difference (B vs A) | The difference between the two variants, displayed as a value and as a percentage. |
| Winning Variant | The variant with the best score for the segment. |
Talon.One shows a segment only when it has enough data and statistical confidence. The table can be missing segments, or appear entirely empty, for these reasons:
- Low traffic: The experiment has fewer than 100 closed sessions per variant, or an individual segment has fewer than 100 sessions per variant.
- Low confidence: A segment does not reach the 95% confidence threshold.
- Too early: The experiment has not run long enough yet. Segments can appear as it collects more sessions.
A dimension is also hidden when its sessions are too similar to split into separate segments.