How Glints uses Unleash for A/B testing

In the beginning of 2020, Glints was in need of an A/B testing platform for us to test out different variants of a feature and we settled on an open source solution - Unleash. We evaluated multiple platforms but decided to use unleash because of the following reasons:

A simple UI for non-tech people to control the experiments
Language agnostic with client libraries in multiple languages
Actively maintained project (we started with Unleash v3 and now we are using Unleash v4 with significant improvement in terms of features)

What is Unleash?

Unleash is an open-source feature management platform. It provides a great overview of all feature toggles/flags across all your applications and services.

Architecture overview:

This is what our architecture with unleash looks like. We have the unleash server which is responsible for storing all the feature toggles, their strategies and their history along with all the unleash user info.

We do not directly access unleash via our front end clients. That would expose unnecessary information about the feature toggles that the user doesn't need to know e.g. which features have been disabled for them. The API server acts as a proxy between the front end clients and the unleash server.

The Glints API server connects to the unleash server to get the data of existing feature toggles. Using this data, the proxy endpoint gives a list of feature flags that have been enabled for the user along with the variant if it exists when requested by the client. Absence of a feature flag from the list should be perceived that the feature flag has been disabled for that user.

The API endpoint would return the feature flags based on the unleash-context provided by the front end client.

Would the unleash server being down affect my application?

Not at all. This is one of the things that we love about Unleash.

All of the Unleash SDKs (unleash-client-node in our case) do something called local evaluation. That means that only the information about a feature toggle is stored on the unleash server, but assigning a user to a variant A or a variant B happens via the SDK. This means that if your unleash server is down, your feature flags would still keep on working on the last known state without any updates possible to the existing state.

Does unleash come with built in tools to analyse the performance of each variant of the A/B test?

No, we use Amplitude at Glints to analyse how each of our variants in the tests are performing. We think its for the best since Unleash is a feature management platform and it does that 1 thing right instead of trying to dip its toes in multiple places.

Checkout https://docs.getunleash.io/ to get started on Unleash!

How Glints uses Unleash for A/B testing

Comments

More from this blog

You should be running your 1-1s, not your manager

An engineer's guide to asking better questions on Slack

Supporting Kebab-Case Attributes And Query Params In Flask-Rest Jsonapi

Modifying Flask-Rest-Jsonapi Exception Handling To Enable Support For Sentry

What is Unleash?

Architecture overview:

Would the unleash server being down affect my application?

Does unleash come with built in tools to analyse the performance of each variant of the A/B test?

Command Palette

Comments

More from this blog

What is Unleash?

Architecture overview:

Would the unleash server being down affect my application?

Does unleash come with built in tools to analyse the performance of each variant of the A/B test?