📘

About this document

This document can be used as a reference to troubleshoot abnormal behavior or other bugs you encounter during the SDK integration process.
For questions about features are supported, implementation know-how or advice, please refer to the Frequently Asked Questions (SDK Integration).

1. An error occurs when calling the variation method with a user-specified identifier

Q. I am using JavaScript SDK. Calling the variation method gives me an error related to the user identifier. We passed the user identifier we created as a parameter, what's the problem?

A. In the case of JavaScript SDK, when passing user information to the variation method, it must be passed in the form of an object. Please check whether only a simple string type is passed as is shown in the wrong case in the example code below.

hackleClient.variation(6, "user_id")       // Incorrect case
hackleClient.variation(6, {id: "user_id"}} // Correct case

👍

How to put custom identifier for each development platform

There are language-specific examples in the Setting up User Identifiers document.

2. Metric anomalies when passing custom identifiers

Q. We have integrated the Hackle SDK and we are using our own customized user identifier to distinguish users. However, the results for our metrics are strange, is there something we should double-check on?

A. Check that both the variation and track methods are passing the user identifier. If either one of the methods passes a user identifier, the other method must also pass the user identifier as well.

// Incorrect case
hackleClient.variation(7, { id : "12345"} ) // User identifier passed directly
hackleClient.track("purchase") // User identifier not passed

// Correct case
// (Case 1) Using customized identifiers
hackleClient.variation(7, { id : "12345"}) 
hackleClient.track("purchase", { id : "12345"})

// (Case 2) Using identifiers managed internally by the SDK
hackleClient.variation(7) 
hackleClient.track("purchase")

In the case of an incorrect case, the user identifiers used in the variation and track methods are different, leading to the incorrect calculation of the metrics.
When distributing users into different test groups and sending events, the same user identifier delivery method must be used to recognize the same user and calculate metrics.
That is, if only one of them passes a user identifier, it is possible that the two methods will perceive it as targeting different users.

3. Errors in the real-time exposure status results

Q. When I call the variation method multiple times for the same user, the numbers reflected in my real-time impressions increase. Is this an error?

A. This is normal. For real-time impressions, this is equivalent to the count or the number of times the 'variation' calls (number of impressions called). That is, the total number of impressions without removing the number of duplicates is the output.
On the other hand, in the case of metric calculations, this is calculated by unique users (UVs) with the duplicates removed. That is, the metrics output is calculated with the number of unique users only.

4. Traffic anomaly after A/B test ends

Q. I finished the A/B test and selected the winning test group of the experiment. However, all the traffic seems to be going to the winning test group even though I wasn't planning to apply the winning test group's logic on my main page yet.

A. Once you select the winning test group of your experiment in the dashboard all the subsequent traffic will automatically be redirected to the winning test group.
Therefore, at the end of the A/B test, please proceed in the following order.

1. Terminate the A/B test and select the winning test group on Hackle Dashboard.
2. Delete the code affiliated with the A/B testing in the production environment.
3. On the Hackle dashboard, click Archive to archive the experiment.

5. Manual assignment of test devices not working properly in JavaScript SDK

Q. I am using the JavaScript SDK. I am trying to manually assign test devices to the desired test group in my development environment, but the traffic distribution does not seem to be working properly.

A. In the case of JavaScript SDK, setting values ​​are cached for user latency. Although it may seem like the traffic is not being properly distributed, you can try the hard refresh function of the Chrome browser in order to have the desired version reflected as soon as possible.

If this doesn't work, please try again by opening the developer tools, right-clicking the refresh button, and then clicking "empty cache" and "hard refresh".

6. Too much traffic is being allocated to test group A

Q. I am currently looking at the real-time exposure status, but the traffic seems to be overly concentrated in Test Group A.

A. Users who are not assigned to an A/B test or are subject to the manual assignment may enter the default group, test group A, for a number of reasons and errors. The reason why users are allocated to test group A regardless of test group distribution is described in the documents [Traffic Distribution Process] (doc:ab-variation) and [Test Group Distribution Principle] (doc:ab-bucketing).

The cause can be identified through the method provided by the SDK. For SDKs that support the variationDetail method, the distribution reason on why a user was allocated to a specific group is also returned. Please make full use of this method.