Mutually Exclusive Group

📘

Please note

Mutually Exclusive Group can be created and managed separately in the production and the development environment.

Overview

If multiple A/B tests are running simultaneously within a service, a user may be exposed to one or more tests.
In general, each test is distributed independently, even if multiple tests are performed simultaneously. For example, if two tests, Test 1 and Test 2, are conducted simultaneously (Test 1: A, Test 2: A), and (Test 1: A, Test 2: B), there is no problem because the distribution of individual experiments is evenly & independently distributed to other experiments. (See Group 1 in the image below)

However, if Test 1 and Test 2 are affecting each other for UX / System reasons or if you want to accurately measure the individual effects of a particular test only, you need to allocate traffic exclusively for each test.

For example, if you run simultaneously Test 1 to change the color of the search button and Test 3 to change the location of the search button, it is difficult to measure the individual effects of each test if you run both tests simultaneously to determine the click rate of the search button.

In this case, you can add A/B tests to the mutual exclusion setting to accurately measure the impact of each test.

In this case, the tests should be mutually exclusive. Test 2 is no longer visible to users who view Test 1, and the percentage of users you set can be exposed to the test to accurately determine the impact of each test.

Create Mutually Exclusive Group

You can create a new Mutually Exclusive Group by clicking the +Create a new Mutually Exclusive Group button in Mutually Exclusive Group menu on the left tab of the A/B Test menu on the dashboard.

Step1. Enter name and description of Mutually Exclusive Group

• Mutually Exclusive Group Name: Required, enter the name of the group you want to create.
• Mutually Exclusive Group Description: Optional, a description will help you understand which Mutually Exclusive A/B tests other team members are conducting.

Step2. Create a Group

Click the 'Add Group' button to create up to 10 independent Mutually Exclusive Groups.

Step3. Set up a Mutually Exclusive Group

1. Group name: Required, enter the name of the Mutually Exclusive Group.
2. Group Traffic Settings: Traffic is a feature that allows only the desired percentage of all users to be exposed to the A/B test. You can set up traffic within the allocatable traffic. (However, the minimum traffic for the group is 10 percent.)
3. Add A/B test: You can add A/B tests to be mutually exclusive. Please be aware that the added tests within the same group are exposed to all users in that group and are not mutually exclusive

Add an A/B test

You can add an existing A/B test in the draft state to a Mutually Exclusive Group

Remove an A/B test

You can click the trash icon to exclude the A/B test at any time.

However, if you exclude an ongoing A/B test from the group, the test exposure will expand from the group to the entire user.
It will affect the result of the experiment because it exposes users other than the initial traffic that you set up.

Below is a screenshot with multiple mutually exclusive groups.
The three groups in the picture are mutually exclusive, and users assigned to one group are no longer exposed to the other group's tests (e.g. users exposed to the Design Team's [Search Window UI Change Test] are no longer exposed to the Marketing Team's [New Subscription Point Test]

• You can modify the group settings by clicking Setup button.
• You can delete a group by clicking the X icon located on the right side of Setup. (However, you can not delete the group to which the A/B test has been added.)

A/B Test Detail Page

If an A/B test is added to the Mutually Exclusive Group, the information of the group will be displayed in the upper area of the A/B test detail page.
You can click the group name to view the details of the group to which the test was added.

Close an A/B test

The Winner group set at the end of the A/B test is only exposed to the user in the Mutually Exclusive Group, not to the entire user.
You can exclude A/B tests from the group to expose the changes to all users.

After closing the test, you can click the group name to go directly to the group details page.
In order to expose the A/B test results to the entire user, the test must be excluded from the group.

📘

To expose A/B tests result to all users, exclude the test from the Mutually Exclusive Group

You can apply changes to all users by excluding the A/B test from the Group