Live Exposure Count
The "Live" tab can be found on the far-right within the page for an individual A/B test.
Here, you can visualize the exposure status of exposed users to a running A/B test by each test group.
- Graph of exposed users for each test group
- Table with information about specific users or device IDs exposed to the A/B test
(Information on the time, user identifier value, and test group are sorted in the order of the most recent 50 cases.)
This graph can be useful as a reference when you want to check whether the traffic distribution to each test group is working properly.
Update
You can only see the live status of the A/B test from within the last 30 minutes, and the graph is updated every 30 seconds. When the update is available, the update button will turn blue, and pressing the button will update both the graph and the table on the dashboard.
Compare the values indicated by the red arrows in the two screenshots below.
After 30 seconds of showing the exposure status, the update button became blue (enabled).
If you click the update button, the status is updated, and the update button will stay gray (inactive) until the 30 seconds pass after the last update.
User Logging
You can check which users were distributed, when they were distributed, to which test group, and the corresponding distribution decision reason of each A/B test. These user logs are available for all users who have been exposed to A/B testing in the last 30 minutes.
| Decision Reason | Description | Distribution Result |
|---|---|---|
| SDK_NOT_READY | The SDK is not ready to use. (e.g. initialized with the wrong SDK key) | (control) Test Group A |
| EXPERIMENT_NOT_FOUND | No A/B tests were found for the experimental key you passed. The experiment key may be incorrect or the experiment may be in the archive status. | (control) Test Group A |
| EXPERIMENT_DRAFT | The A/B test is ready to start. | (control) Test Group A |
| EXPERIMENT_PAUSED | The A/B test has been paused | (control) Test Group A |
| EXPERIMENT_COMPLETED | A/B test has ended. | Final winning test group from the experiment |
| OVERRIDDEN | Users are distributed to a specific test group by manual assignment. | Manually assigned test group |
| TRAFFIC_NOT_ALLOCATED | A/B test is running, but user has not been assigned to the experiment. | (control) Test Group A |
| TRAFFIC_ALLOCATED | User has been assigned to A/B test. | Assigned test group |
| VARIATION_DROPPED | The test group was removed from the A/B test. | (control) Test Group A |
| EXCEPTION | An unknown error has occurred. | (control) Test Group A |
Criteria for Counts
Real-time exposure stats show the total number of exposures without removing duplicates.
The number of unique users without duplicates can be seen in the Metrics tab.
Updated about 2 years ago