# Manage experiments

> After activating an experiment, you can monitor its performance, manage its state, and evaluate results. Managing an experiment's state works in a similar way to managing a campaign's state.

> For the complete documentation index, see [llms.txt](https://docs.talon.one/llms.txt).

An experiment's schedule affects its state similarly to that of a
[campaign schedule](/docs/product/campaigns/settings/manage-campaign-schedule.md).

:::note
Be aware of overlap when running multiple experiments or campaigns that target the same
customer group. Overlapping promotions can dilute the statistical significance of your
results, making it difficult to distinguish which incentive is driving customer behavior.
:::

## Evaluate experiment results

You can monitor the performance of an experiment in the experiment dashboard.

To open the experiment dashboard:

1. From the left-side menu, click <Experiment className="icon"/> **Experiments** to open
   the experiment list.
1. Click the name of an experiment.
1. On the left-side menu of the experiment, click <Dashboard className="icon"/>
   **Dashboard**.

To the right of the experiment's name, the experiment's state is displayed, for example,
<StateLabel type="running" name="Running" />.

:::note
While Talon.One provides data-driven recommendations, we recommend reviewing the detailed
metrics before making business decisions. For example, you can compare metrics, such as
revenue gains versus discount costs.
:::

The following details are shown below the experiment name, along with an AI-generated
[summary](#understand-the-summary):

- **Experiment ID**: The ID of the experiment.
- **Campaign ID**: The ID of the campaign that was generated upon experiment creation.
- **Created**: The date and time when the experiment was created.
- **Start date**: The date and time when the experiment starts, as set under **Schedule**.
- **End date**: The date and time when the experiment ends, as set under **Schedule**.

### Understand the summary

The summary is an AI-generated explanation of your experiment's results.

The AI model that generates the summary analyzes the metrics in the
[evaluation table](#understand-the-evaluation-table) that express averages, and the
[confidence scores](#evaluate-confidence-scores) for these metrics. It reports whether there's a winning
variant, a trade-off between variants, or an inconclusive result. It also highlights any
metrics with a significant change, and can provide recommendations based on the
experiment's outcome.

:::note
- Talon.One enables the AI-generated summary by default. If your organization's policy
  does not allow AI-powered features, contact your Customer Success Manager to disable
  this feature.
- The AI model behind the summary only receives the metrics and confidence scores shown in
  the evaluation table. It does not receive individual order values or personally
  identifiable information.
:::

Use the summary to, for example, decide whether to
[roll out a winning variant as a campaign](/docs/product/campaigns/experiments/create-campaigns-from-experiments.md).
You can also use it to understand the experiment's outcome without analyzing the data
yourself.

To ensure your summary reflects the latest data collected for your experiment, click
<AI className="icon"/> **Regenerate**. This option becomes available after your
experiment has collected enough data.

### Understand the evaluation table

The evaluation table displays the metrics and values used to evaluate your experiment's
performance and generate data-driven recommendations. It includes the following columns:

| Column              | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Metrics             | The metrics that Talon.One evaluates for the experiment. For a description of each one, see [Experiment metrics](#experiment-metrics).                                                                                                                                                                                                                                                                                                                                      |
| Confidence          | Values that indicate whether the performance difference between Variant A and Variant B is a statistically significant and repeatable result, or is due to random chance. The confidence score values are calculated using Welch's two-sided t-test and displayed as percentages.**Note**: You can only view the confidence score for metrics that express averages: **Gross average order value (AOV)**, **Net average order value (AOV)**, and **Average units per order (UPO)**. For the other metrics, this column shows `N/A`. To see how these scores have changed over the duration of the experiment, refer to the [**Confidence** chart](#evaluate-confidence-scores) below the evaluation table. |
| Variant A/B         | The respective variant's values for the data points listed under **Metrics**. These values are displayed as either currency or integers, depending on the metric type. The experiment's [goal](/docs/product/campaigns/experiments/experiment-goals.md) determines the winning variant. When the goal sets a primary metric, the variant that reaches statistical significance on that metric is displayed as the **Winning variant**. When the **Other** goal type is used, if one variant has a best overall score, that variant is displayed as the **Winning variant**.                                                                                                                                                                                            |
| Difference (B vs A) | The difference in results between the variants. In the left column, the difference values are displayed as either currency or integers, depending on the metric type. In the right column, all values are displayed as percentages.**Note**: If **Variant A** has the best overall score, these columns appear next to the **Variant A** column and their name is shown as **Difference (A vs B)**.                                                             |

### Experiment metrics

The evaluation table lists the following metrics for each variant:

| Metric | Description |
| --- | --- |
| **Gross average order value (AOV)** | The average gross revenue from orders with at least one effect applied. |
| **Net average order value (AOV)** | The average net revenue from orders with at least one effect applied. |
| **Average units per order (UPO)** | The average number of items purchased per order with at least one effect applied. |
| **Orders** | The total number of closed sessions with at least one effect applied. |
| **Gross revenue** | The total pre-discount value of orders with at least one effect applied. |
| **Discounts** | The total value of applied discounts from all orders. |
| **Coupons** | The total number of redeemed coupons from all orders. |

### Evaluate confidence scores

The **Confidence** chart shows how the confidence score values for your experiment have
changed over its duration. Use it to check whether your results have stabilized before you
decide to stop the experiment.

:::note
The **Confidence** chart appears only after the experiment has collected enough data.
:::

The chart displays a trend line for each of the three
[metrics](#experiment-metrics) that express averages.

The confidence score, shown as a percentage, is tracked over the duration of the
experiment starting on the start date, with a dashed line marking the 90% threshold that
Talon.One recommends reaching for reliable results. For a running experiment, the
confidence score values ends on the current date. For a disabled experiment, it ends on
the date the experiment was disabled.

To see the confidence scores for a specific date, hover over the chart.

Use the shape of the trend lines to decide whether to stop the experiment:

- If the confidence scores have leveled off above the 90% threshold, your results are
  stable and you can stop the experiment.
- If the confidence scores are still climbing, let the experiment continue until they
  stabilize.
- If the confidence scores fluctuate, your results are not yet reliable. Let the
  experiment run for longer to gather more data.

### Evaluate segment insights

The **Segment insights** table finds customer segments where one variant outperforms the
other for a specific metric, with high confidence. This can reveal groups of customers that
behave in a particular way, even when the overall experiment result is inconclusive.

To learn more about the available segments and how to read the table, see
[Segment insights](/docs/product/campaigns/experiments/segment-insights.md).

## Related pages

- [Create experiments](/docs/product/campaigns/experiments/create-experiments.md)
- [Experiment goals](/docs/product/campaigns/experiments/experiment-goals.md)
- [Segment insights](/docs/product/campaigns/experiments/segment-insights.md)
- [Campaigns](/docs/product/campaigns/overview.md)
- [Create campaigns from experiments](/docs/product/campaigns/experiments/create-campaigns-from-experiments.md)
- [Application dashboard](/docs/product/campaigns/analytics/application-dashboard.md)