There are several statuses the experiment can undertake during A/B testing.
The dashboard shows the current status of A/B tests and allows you to start, pause, end, and archive the experiments.
Prior to initiating a live A/B test by pressing start, it will be marked as 'Draft'.
The diagram below outlines all the A/B test statuses possible depending on the environment.
With Create an A/B Test a new A/B Test is created for use in both the production and development environments.
- 5 different statuses,
- Status can be changed.
- When changing the status of the experiment to the
Archivedstatus, the status of the experiment in the development environment will also be changed.
- 3 different statuses
- At the same time as the A/B test is created, it is automatically in the
- The status cannot be changed in this environment and it follows the status set in the production environment. That is, when the experiment in the production environment enters the end state, the development environment also enters the end state.
Running tests in multiple environments
A/B testing can be carried out simultaneously in the development environment and the production environment. It is also possible to conduct A/B testing in just one of the two environments.
However, remember that if you end an A/B test in the production environment, the experiment will also end in the development environment.
It covers how to change the status of A/B tests in the dashboard.
As described above, status changes are only possible in the production environment.
You can start by clicking the
Start button located at the top right of the A/B test page. After setting the traffic allocation level in the pop-up window that is displayed and clicking the
Start button again, users are exposed to the A/B test according to your traffic allocation level.
Pause button located at the top right of the A/B test page. All users of paused A/B testing will be exposed to the previous current version of the service.
End button located at the top right of the A/B test page. In the pop-up window, select the winning test group and click the
Winning Test Group
After the Winner group is selected after ending the experiment, users will no longer be distributed and all the following users will be allocated to the page/logic version provided to Group B. For example, if the Winner group is selected as Test Group B, then all users from that point onwards will be exposed to the logic corresponding to the version provided to Test Group B.
Click on 'Archive' and Hackle will preserve the A/B test results.
Archive exists to prevent terminated A/B tests from generating unnecessary server traffic.
You can proceed by clicking the
Archive button located at the top right of the page for the terminated A/B test. Please remove any code related to A/B testing before archiving (developer should get involved during this step).
Updated about 1 year ago