Below is a list of the terms frequently used. These are terms that will frequently appear on the Hackle dashboard and workspace.

📘

For developers

Several “key” values are required in the SDK integration/programming process. These programming language-related terms will be accompanied by the term “For developers.”

General

Dashboard

Hackle’s platform and general workspace for users to interact with Hackle’s services.

Workspace

A shared space for a corporation (or team). All actions/services provided by Hackle including running an A/B test, funneling, and creating/managing events will occur in this shared space. You can share previously set A/B tests, funnels, and events within the same workspace.

Typically, one company uses one workspace. However, teams or service units within a company can each have their own workspaces.

Environment

There are multiple environments that are provided by Hackle’s workspace. The first environment is the production environment which deals with delivering the experimentation to actual consumers of the platform. The second environment is the development environment that can be used during developmental stages and conducting quality assurance before the actual launch of new A/B tests or feature flags for you real consumers.

User Identifier (or Device ID)

The value or a means of identification assigned to individual users. The value must be a unique value without duplicates for each user.
For more details, please refer to Managing User Identifiers

SDK (Software Development Kit)

A set of development tools that can be integrated into your website or mobile app in order to carry out the A/B test experimentation, roll out new features designated for different test groups, and collect results for data analysis. In other words, think of the SDK as a prerequisite code that allows your business to use our SAAS services inside your online service channel.

Users can choose the SDK that corresponds to the programming language used to create the website or app.

SDK Key

For developers This is the key needed to classify different environments when integrating SDK.

---

A/B Test (Experiment)

Simply put, an A/B test is a controlled experiment where you can compare different versions of a page to users at random through testing the new features, screen display, designs, algorithms, etc. against the current features, screens, and algorithms. The terms A/B test and experiment are used interchangeably.

Test Groups/Different Versions & Variants of the Page, Feature, etc.

The test groups are basically the different groups that are exposed to the versions (features, screens, algorithms, etc) of a page of an experiment. Test groups include both the control Test Group A and treatment Test Groups B, C, D, etc…
Simply put, those allocated to control Test Group A are exposed to the original page, and the users allocated to an alternative Test Group, say, Group B will be exposed to a different version of the same page. In every Hackle experiment, Group A will always be set as the control group by default and will always be required in order to benchmark the impact of a certain variation of a page. Up to 10 Test Groups (1 control Test Group A and 9 other versions of Test Group B,C,D… ) can be created with Hackle.

Example)
A/B test experimenting with the color color of the “purchase button” of a page

Group A: Original color (blue)
Group B: change color (red)
Group C: change color (green)

A/B test experimenting with the change in the search algorithm

Group A: Original Algorithms
Group B: Modified Algorithms

Metrics (Goals)

“Metrics” will need to be set to measure and compare the performance of each group of the A/B tests and to see whether or not the versions shown to the Test Groups (B,C,D..) led to better results than the control Test Group A. The desired metrics can be easily created to measure for each A/B test from the Hackle dashboard

For example, if you set the 'purchase conversion rate' as a key goal in conducting the A/B test for determining the color of the purchase button, you can set your end metrics as 'purchase conversion rate' or 'number of purchase button clicks'

Traffic Distribution

Users must be assigned or distributed to one of the test groups. The allocation of a user to a certain test group is determined by a series of distribution processes.

Unless the conditions of the A/B test change, users are always assigned to the same test group. In other words, for an experiment that tests different colors of the purchase button, if user Emma Smith is allocated to Group B, Emma Smith will continue to belong to Group B until the end of the experiment.

For further information on the distribution processes please refer to the Principles behind Traffic Distribution document and for a deeper understanding of the principles behind our distribution mechanisms.

Traffic Allocation

Traffic Allocation refers to the percentage of total users who will be subject to the A/B test.

For example, if you set the traffic allocation level to 40%, only 40% of all users will be subjected to the A/B test and the remaining 60% will not be tested. Hence the results of the A/B testing with only data from 40% of users assigned to A/B testing.

Traffic Allocation and Traffic Distribution are two very different concepts.

In the example above, if the traffic allocation level is set to 40%, users will be distributed to each test group according to the traffic distribution ratio. With Hackle, traffic is distributed evenly (50:50) to each of the two test groups from the traffic allocation of 40%.

Manual Allocation (Override)

Through manual allocation, you are able to force the assignment of certain users (or test devices) to a specific test group. Users subject to manual allocation are assigned to the specified test group regardless of traffic allocation or test group distribution ratio.

This feature is primarily used to check the behavior of a specific test group during experiments conducted on the development environment and to carry out QAs before starting the experiment on real users.

Test Devices

Test devices are basically the designated users to test the experiment with during manual allocation. These "test" devices can be in the form of user identifiers, device IDs, or basically anything that defines a specific user.

Targeting

This is a function that you can set to control and specify the users with certain properties to be exposed to the A/B test or feature flag. For example, the participants of the A/B test can be targeted or geared towards only users using Android mobile operating systems.
However, a user who is included in manual allocation cannot be a part of targeting.

Experiment Key

Indicator to distinguish between individual A/B tests. The experiment key is automatically issued when you create an A/B test.

For developers experiment key is used when distinguishing between experiments in order to distribute user traffic into different test groups.

---

Data Segmentation Analytics

If you include properties in the user object and send them to the Hackle, you can do a more detailed analysis based on the properties.
For example, if you sent a property called 'Membership Level', you can analyze the data by classifying users according to their membership level.

Segementation properties

The target criteria for detailed analysis. It can be viewed as a group of classes or ranges with the same properties.
For example, if you say "analyze by the OS used by each user", the analysis standard or segmentation property becomes the OS.

Property value

The specific value selected for analysis among the possible values ​​of the specific segmentation property.
For example, if you select OS as the segmentation property, you can put Android, iOS, Windows, MacOS, etc. as your property values.

Property

It is a method of expressing the characteristics of a certain user with property name (key) and property value (value).
For example, if you want to express the operating system as a property, the property name can be os, and the property value can include Android, iOS, Windows, MacOS, etc.

---

Feature Flag

A feature flag is a software development method that allows developers to remotely enable or disable features without deploying code. This toggling or flagging method can be useful to separate code deployment and feature release.

Feature Key

A key used to identify individual feature flags. Feature keys are automatically issued when a new feature flag is generated.
For developers A feature key is a unique value within the workspace and is used during the feature flag determination phase.

---

Funnel

A funnel is the hypothesized path that a user takes in successive orders on your website. Funnel analysis is an analytics tool for visualizing the user journey within your service or product with data. Through a funnel analysis, you are able to understand how users navigate through a designated path or where in the path users drop off at. Through the funnel analysis, you can check how many users have entered the subsidiary screen at the later stages of the journey compared to the main landing page of your product and check how the data is changing on a daily/weekly/monthly/quarterly basis.

---

Event

Event is an action or event that a user fires while he or she is using a website, mobile application, system, etc. User actions include clicking the search button, entering a product page, and completing a purchase. Other events that do not necessarily involve a user "acting upon" something but rather events fired from the user just being on the page include order book loading time, server API processing time, and mobile app crashes.

In the case of Hackle, events are required to handle the following:

• Compiling data for calculating the results of a metric during an A/B test
• Receiving data for funnel analysis

Event Key

An event key is compulsory when creating a new event. Not only is the key required to identify and set the event apart from other events, but it is required for when Hackle collects to log events back to the dashboard.

For developers The event key is a unique value within the workspace and is used as a parameter for when the SDK tracks and sends user events (once the SDK integration is complete).