alephbet

by Alephbet

Alephbet / alephbet

AlephBet is a pure-javascript A/B (multivariate) testing framework for developers.

243 Stars 18 Forks Last release: about 1 year ago (v0.18.0) 126 Commits 37 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

AlephBet

Build Status

AlephBet is a pure-javascript A/B (multivariate) testing framework for developers.

Key Features:

  • NEW: run your own tracking backend on Rails with the alephbet rubygem (still experimental)
  • NEW: run your own tracking backend on AWS Lambda with Lamed (recommended)
  • Weighted variants. See https://github.com/Alephbet/alephbet/pull/20
  • user-based / cross-device experiments. See https://github.com/Alephbet/alephbet/issues/16
  • run your own tracking backend on AWS Lambda with Gimel (recommended)
  • Pluggable backends: event tracking (defaults to Google Universal Analytics), and storage (defaults to localStorage)
  • Supports multiple variants and goals
  • Tracks unique visitors and goal completions
  • Flexible triggers
  • Ideal for use with page and fragment caching
  • Developer-friendly for both usage and contirbution (using npm / webpack)

What does AlephBet mean?

Aleph (אלף) Bet (בית) are the first two letters in the Hebrew alphabet. Similar to A and B.

Inspiration

AlephBet was heavily inspired by Optimizely (sans WYSIWYG and reporting) and Cohorts.js. The code structure and some code elements were taken from cohorts.js, with some notable changes to terminology and built-in support for unique goals and visitor tracking.

For more detailed info about the background and rationale for creating AlephBet, please check out this blog post

Screencast

AlephBet Screencast

code snippet used on the screencast

for more screencasts, tips and info, please check the wiki

Quick Start

  • Make sure your Google Universal analytics is set up.
  • Download and include
    alephbet.min.js
    in the head section of your HTML.
  • Create an experiment:
var button_color_experiment = new AlephBet.Experiment({
  name: 'button color',  // the name of this experiment; required.
  variants: {  // variants for this experiment; required.
    blue: {
      activate: function() {  // activate function to execute if variant is selected
        $('#my-btn').attr('style', 'color: blue;');
      }
    },
    red: {
      activate: function() {
        $('#my-btn').attr('style', 'color: red;');
      }
    }
  },
});
  • Track goals for your experiment:
// creating a goal
var button_clicked_goal = new AlephBet.Goal('button clicked');
$('#my-btn').on('click', function() {
  // The chosen variant will be tied to the goal automatically
  button_clicked_goal.complete();
});

// adding experiment to the goal button_clicked_goal.add_experiment(button_color_experiment);

// alternatively - add the goal to the experiment button_color_experiment.add_goal(button_clicked_goal);

// tracking non-unique goals, e.g. page views var page_views = new AlephBet.Goal('page view', {unique: false});

  • view results on your Google Analytics Event Tracking Section. The experiment name + variation will be assigned to

    actions
    , and Visitors or Goals to
    label
    . e.g.
    • action:
      button color | red
      , label:
      Visitors
      : unique count of visitors assigned to the
      red
      variant.
    • button color | blue
      ,
      button clicked
      : unique visitors clicking on the button assigned to the
      blue
      variant.
    • button color | red
      ,
      viewed page
      : count of pages viewed by all visitors (not-unique) after the experiment started.
  • important note: whilst Google Analytics is the easiest way to start playing with Alephbet, it's definitely not the best way to use it. GA starts sampling events after you reach a certain volume, and the built-in GA adapter does not support more advanced features like cross-device tracking. If you're serious about running A/B tests, I would urge you to consider using Lamed, Gimel or another backend instead.

Advanced Usage

Recommended Usage Pattern

AlephBet was meant to be used across different pages, tracking multiple goals over simultaneous experiments. It is therefore recommended to keep all experiments in one javascript file, shared across all pages. This allows sharing goals across different experiments. Experiments can be triggered based on a set of conditions, allowing to fine-tune the audience for the experiments (e.g. mobile users, logged-in etc).

Triggers

Experiments automatically start by default. However, a trigger function can be provided, to limit the audience or the page(s) where the experiment "kicks-off". ```javascript var buttoncolorexperiment = new AlephBet.Experiment({ name: 'button color', trigger: function() { return window.location.href.match(/pricing/); }, variants: { // ... }, });

// triggers can be assigned to a variable and shared / re-used var loggedinuser = function() { return document.cookie.match(/_session/); }; var mobilebrowser = function() { // test if mobile browser };

var bigheaderexperiment = new AlephBet.Experiment({ name: 'big header', trigger: function() { return loggedinuser() && mobile_browser(); }, // ... }); ```

NOTE: once a user participates in an experiment, the trigger is no longer checked. See #9

Sample size

You can specify a

sample
float (between 0.0 and 1.0) to limit the number of visitors participating in an experiment.

Weights

Whilst

sample
will limit the entire experiment to a subset of potential participants,
weight
allows you to apply a weighted-random selection of variants. This can be considered a first step (manual) way to implement Multi Armed Bandit testing.

NOTE: Weights can be any integer value. Do not use floats. You can use any number, but it's probably easiest to treat it as a percentage, e.g. use weights of 80, 20 to allocate ~80% to one variant vs. ~20% to the other.

var button_color_experiment = new AlephBet.Experiment({
  name: 'button color',  // the name of this experiment; required.
  variants: {  // variants for this experiment; required.
    blue: {
      activate: function() {  // activate function to execute if variant is selected
        $('#my-btn').attr('style', 'color: blue;');
      },
      weight: 50 // optional, can be any integer value
    },
    red: {
      activate: function() {
        $('#my-btn').attr('style', 'color: red;');
      },
      weight: 50
    }
  },
});

Visitors

Visitors will be tracked once they participate in an experiment (and only once). Once a visitor participates in an experiment, the same variant will always be shown to them. If visitors are excluded from the sample, they will be permanently excluded from seeing the experiment. Triggers however will be checked more than once, to allow launching experiments under specific conditions for the same visitor.

User-based / Cross-device tracking

You can now pass a

user_id
to the experiment as an optional parameter. This allows experiment to work across devices on a per-user basis.
var button_color_experiment = new AlephBet.Experiment({
  name: 'button color',
  user_id: get_user_id(),  // pass over the unique user id bound to this experiment
  trigger: function() {
    // do not trigger this expeirment without a user_id
    return get_user_id() && other_condition();
  },
  variants: {  // variants for this experiment; required.
    blue: {
      activate: function() {  // activate function to execute if variant is selected
        $('#my-btn').attr('style', 'color: blue;');
      }
    },
    red: {
      activate: function() {
        $('#my-btn').attr('style', 'color: red;');
      }
    }
  },
});

// do not assign goals without a user_id if (get_user_id()) { button_color_experiment.add_goal(my_goal); }

Notes:

  • For user-based tracking, make sure you always have a user_id. Do not mix visitors (without an id) and users (with an id) in the same experiment.
  • Cross-device tracking only works with the Gimel, Lamed or keen.io tracking backends. It does not work with Google Analytics.

See this Wiki page for more information

Goals

Goals are uniquely tracked by default. i.e. if a goal is set to measure how many visitors clicked on a button, multiple clicks won't generate another goal completion. Only one per visitor. Non-unique goals can be set by passing

unique: false
to the goal when creating it.

Goals will only be tracked if the experiment was launched and a variant selected before. Tracking goals is therefore safe and idempotent (unless unique is false).

Here's a short sample of tracking multiple goals over multiple experiments:

// main goal - button click
var button_click_goal = new AlephBet.Goal('button click');
$('#my-btn').on('click', function() {
  button_clicked_goal.complete();
});

// engagement - any click on the page var engagement = new AlephBet.Goal('engagement'); $('html').on('click', function() { engagement.complete(); });

var all_goals = [button_click_goal, engagement];

// experiments var button_color_experiment = new AlephBet.Experiment({ /* ... / }); var buy_button_cta_experiment = new AlephBet.Experiment({ / ... */ });

// adding all goals to experiments _(all_goals).each(function (goal) { button_color_experiment.add_goal(goal); buy_button_cta_experiment.add_goal(goal); });

// alternatively, you can use the add_goals method and pass it an array of goals button_color_experiment.add_goals(all_goals); buy_button_cta_experiment.add_goals(all_goals);

Custom Tracking Adapter

AlephBet comes with a built-in Google Analytics adapter and three, currently experimental, adapters with potentially better accuracy:

Persistent Queue GA Adapter

Persistent Queue Keen Adapter

Gimel adapter

Lamed adapter

Creating custom adapters is however very easy.

Here's an example for integrating an adapter for keen.io

(For a more complete implementation, you should use the built-in

Alephbet.PersistentQueueKeenAdapter
)

Custom Storage Adapter

Similar to the tracking adapter, you can customize the storage adapter. AlephBet uses localStorage by default, but if you want to use cookies or customize how data is persisted on the client, creating an adapter is very easy.

Here's a simple example of a cookie storage adapter with expiry of 30 days, using js-cookie:

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.