Step 2-1: Set Retention Metric

🚧

Set Metric is only available in Production Environment

A/B Test Retention

You can check the retention chart for each group of exposed users to the test. In addition, you can compare the improvement rate and p-value of the treatment group (group B) compared to the control group (group A), which gives you insight for test result analysis.

Sets Metrics

You can start retention analysis by clicking the 'Create Retention' button in the Retention Tab. You can set up to five metrics.

1. Name and Description
When you click 'Create Retention', you need to enter the name, description and set the event of the retention

2. Set Retention: Start Event and Return Event
You can calculate the retention rate as the number of user who caused a return event within a time frame divided by the number of user who caused a start event.

• Start Event: The user event defines the user cohort for retention analysis.
  • A/B test first exposure : The entire unique user exposed to the A/B test. All of user exposed to the A/B test first time will be counted even without any further actions
• Adding additional Start Event : The user who caused a certain event after their first A/B test exposure will be only counted.
• You can use the filter function for the first exposure event or additional event after the exposure.
• Adding a Start Event feature is supported for Pro plan and above.
• Return Event : The return event is a criteria to determine how many of the users in the cohort retained
  • You can use the filter function for a return event.
  • You can set 'Any Event' for a return event. All of active user event that sent to Hackle. You can easily check the site-wide revisit rate by selecting Any Event.

The above setting means the retention rate for Any Event of the user group that was exposed to the A/B test. The setting is illustrated in the diagram below.