Traffic Distribution

When conducting an A/B test, not only do you need to distribute your users to different test groups, you also need to write a logic for each test group. After the integration stages, the distribution of users into different test groups can be done through the Hackle SDK.

📘

Test Groups

The test groups are basically the different groups that are exposed to the versions (features, screens, algorithms, etc) of a page of an experiment. Test groups include both the control test group A and treatment test groups B, C, D, etc., that are exposed to the "improved" versions of the pages.
The test groups can be set on the dashboard and further information on the test groups can be found on the What is a Test Group? document.

variation

Think of the variation as the term that is equivalent to different versions of the page being shown to users on your app or browser.
By passing the experiment key to the variation() method, you can distribute users and receive the result for the specific experiment. After that, we implement the logic of the page version applicable for each test group.

In the example code below, we are passing an experiment key of 42, and there are two test groups, A and B.

// Determines the test group to assign the user in an A/B tests with an experiment key of 42.
// For undetermined cases, the user is returned to test group A.
let variation = hackleApp.variation(experimentKey: 42)

// Logic for the different test groups. 
if variation == "A" {
  // Logic for test group A
} else if variation == "B" {
  // Logic for test group B
}
// Determines the test group to assign the user in an A/B tests with an experiment key of 42.
// For undetermined cases, the user is returned to test group A.
NSString *variation = [app variationWithExperimentKey:42 defaultVariation:@"A"];

// Logic for the different test groups. 
if([variation isEqual:@"A"]) {
    // Logic for test group A
} else if ([variation isEqual:@"B"]) {
    // Logic for test group B
}

variationDetail

The variationDetail() method works the same as the variation() method but provides the reason behind why a user is distributed to a specific group. This method can be a useful tool to see if the distribution is working properly.
You must pass the experiment key as a parameter. For the example code below, we are passing experimental key 42.

// Traffic distribution details 
let decision: Decision = hackleApp.variationDetail(experimentKey: 42)

// Test group determined from traffic distribution
let variation: String = decision.variation

// Reason for distribution to a specific test group
let reason: String = decision.reason
// Traffic distribution details 
Decision *decision = [hackleApp variationDetailWithExperimentKey:@42 defaultVariation:@"A"];

// Test group determined from traffic distribution
NSString *variation = [decision variation];

// Reason for distribution to a specific test group
NSString *reason = [decision reason];

You will receive the reason for the distribution or the allocation of a specific user to a specific test group in the form of SDK_NOT_READY. Please refer to the table below for the full list of different distribution reasons.

Reason

Description

Distribution Result

SDK_NOT_READY

The SDK is not ready to use.
(e.g. initialized with the wrong SDK key)

(control) Test Group A

EXPERIMENT_NOT_FOUND

No A/B tests were found for the experimental key you passed.
The experiment key may be incorrect or the experiment may be in the archive status.

(control) Test Group A

EXPERIMENT_DRAFT

The A/B test is ready to start.

(control) Test Group A

EXPERIMENT_PAUSED

The A/B test has been paused.

(control) Test Group A

EXPERIMENT_COMPLETED

A/B test has ended.

Final winning test group from the experiment

OVERRIDDEN

Users are distributed to a specific test group by manual assignment.

Manually assigned test group

NOT_IN_EXPERIMENT_TARGET

User does not qualify as the target user for the A/B test.

(control) Test Group A

TRAFFIC_NOT_ALLOCATED

The user is qualified as a target user of the A/B test but is not included as a part of the traffic.

(control) Test Group A

TRAFFIC_ALLOCATED

User has been assigned to A/B test.

Assigned test group

VARIATION_DROPPED

The test group was removed from the A/B test.

(control) Test Group A

EXCEPTION

An unknown error has occurred.

(control) Test Group A


Did this page help you?