Need help with alephbet?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

Alephbet
263 Stars 20 Forks 173 Commits 1 Opened issues

Description

AlephBet is a pure-javascript A/B (multivariate) testing framework for developers.

Services available

!
?

Need anything else?

Contributors list

AlephBet

AlephBet is a pure-javascript A/B (multivariate) testing framework for developers.

Key Features:

  • NEW: run your own tracking backend on Rails with the alephbet rubygem (still experimental)
  • NEW: run your own tracking backend on AWS Lambda with Lamed (recommended)
  • Weighted variants. See https://github.com/Alephbet/alephbet/pull/20
  • user-based / cross-device experiments. See https://github.com/Alephbet/alephbet/issues/16
  • run your own tracking backend on AWS Lambda with Gimel (recommended)
  • Pluggable backends: event tracking (defaults to Google Universal Analytics), and storage (defaults to localStorage)
  • Supports multiple variants and goals
  • Tracks unique visitors and goal completions
  • Flexible triggers
  • Ideal for use with page and fragment caching
  • Developer-friendly for both usage and contirbution (using npm / webpack)

What does AlephBet mean?

Aleph (אלף) Bet (בית) are the first two letters in the Hebrew alphabet. Similar to A and B.

Inspiration

AlephBet was heavily inspired by Optimizely (sans WYSIWYG and reporting) and Cohorts.js. The code structure and some code elements were taken from cohorts.js, with some notable changes to terminology and built-in support for unique goals and visitor tracking.

For more detailed info about the background and rationale for creating AlephBet, please check out this blog post

Screencast

AlephBet Screencast

code snippet used on the screencast

for more screencasts, tips and info, please check the wiki

Quick Start

  • Make sure your Google Universal analytics is set up.
  • Download and include
    alephbet.min.js
    in the head section of your HTML.
  • Or import it in your javascript code
import AlephBet from "alephbet" # ES6 module syntax
const AlephBet = require("alephbet") # commonJS syntax
  • Create an experiment:
import AlephBet from "alephbet";

const button_color_experiment = new AlephBet.Experiment({ name: 'button color', // the name of this experiment; required. variants: { // variants for this experiment; required. blue: { activate: function() { // activate function to execute if variant is selected $('#my-btn').attr('style', 'color: blue;'); } }, red: { activate: function() { $('#my-btn').attr('style', 'color: red;'); } } }, });

  • Track goals for your experiment:
// creating a goal
const button_clicked_goal = new AlephBet.Goal('button clicked');
$('#my-btn').on('click', function() {
  // The chosen variant will be tied to the goal automatically
  button_clicked_goal.complete();
});

// adding experiment to the goal button_clicked_goal.add_experiment(button_color_experiment);

// alternatively - add the goal to the experiment button_color_experiment.add_goal(button_clicked_goal);

// tracking non-unique goals, e.g. page views const page_views = new AlephBet.Goal('page view', {unique: false});

  • view results on your Google Analytics Event Tracking Section. The experiment name + variation will be assigned to

    actions
    , and Visitors or Goals to
    label
    . e.g.
    • action:
      button color | red
      , label:
      Visitors
      : unique count of visitors assigned to the
      red
      variant.
    • button color | blue
      ,
      button clicked
      : unique visitors clicking on the button assigned to the
      blue
      variant.
    • button color | red
      ,
      viewed page
      : count of pages viewed by all visitors (not-unique) after the experiment started.
  • important note: whilst Google Analytics is the easiest way to start playing with Alephbet, it's definitely not the best way to use it. GA starts sampling events after you reach a certain volume, and the built-in GA adapter does not support more advanced features like cross-device tracking. If you're serious about running A/B tests, I would urge you to consider using Lamed, Gimel or another backend instead.

Advanced Usage

Recommended Usage Pattern

AlephBet was meant to be used across different pages, tracking multiple goals over simultaneous experiments. It is therefore recommended to keep all experiments in one javascript file, shared across all pages. This allows sharing goals across different experiments. Experiments can be triggered based on a set of conditions, allowing to fine-tune the audience for the experiments (e.g. mobile users, logged-in etc).

Triggers

Experiments automatically start by default. However, a trigger function can be provided, to limit the audience or the page(s) where the experiment "kicks-off". ```javascript import AlephBet from "alephbet";

const buttoncolorexperiment = new Alephbet.Experiment({ name: 'button color', trigger: () => { return window.location.href.match(/pricing/); }, variants: { // ... }, });

// triggers can be assigned to a variable and shared / re-used const loggedinuser = function() { return document.cookie.match(/_session/); }; const mobilebrowser = function() { // test if mobile browser };

const bigheaderexperiment = new Alephbet.Experiment({ name: 'big header', trigger: () => { return loggedinuser() && mobile_browser(); }, // ... }); ```

NOTE: once a user participates in an experiment, the trigger is no longer checked. See #9

Sample size

You can specify a

sample
float (between 0.0 and 1.0) to limit the number of visitors participating in an experiment.

Weights

Whilst

sample
will limit the entire experiment to a subset of potential participants,
weight
allows you to apply a weighted-random selection of variants. This can be considered a first step (manual) way to implement Multi Armed Bandit testing.

NOTE: Weights can be any integer value. Do not use floats. You can use any number, but it's probably easiest to treat it as a percentage, e.g. use weights of 80, 20 to allocate ~80% to one variant vs. ~20% to the other.

import {Experiment} from "alephbet"

const button_color_experiment = new Experiment({ name: 'button color', // the name of this experiment; required. variants: { // variants for this experiment; required. blue: { activate: function() { // activate function to execute if variant is selected $('#my-btn').attr('style', 'color: blue;'); }, weight: 50 // optional, can be any integer value }, red: { activate: function() { $('#my-btn').attr('style', 'color: red;'); }, weight: 50 } }, });

Visitors

Visitors will be tracked once they participate in an experiment (and only once). Once a visitor participates in an experiment, the same variant will always be shown to them. If visitors are excluded from the sample, they will be permanently excluded from seeing the experiment. Triggers however will be checked more than once, to allow launching experiments under specific conditions for the same visitor.

User-based / Cross-device tracking

You can now pass a

user_id
to the experiment as an optional parameter. This allows experiment to work across devices on a per-user basis.
import AlephBet from "alephbet";

const button_color_experiment = new Alephbet.Experiment({ name: 'button color', user_id: get_user_id(), // pass over the unique user id bound to this experiment trigger: () => { // do not trigger this expeirment without a user_id return get_user_id() && other_condition(); }, variants: { // variants for this experiment; required. blue: { activate: function() { // activate function to execute if variant is selected $('#my-btn').attr('style', 'color: blue;'); } }, red: { activate: function() { $('#my-btn').attr('style', 'color: red;'); } } }, });

// do not assign goals without a user_id if (get_user_id()) { button_color_experiment.add_goal(my_goal); }

Notes:

  • For user-based tracking, make sure you always have a user_id. Do not mix visitors (without an id) and users (with an id) in the same experiment.
  • Cross-device tracking only works with the Gimel, Lamed or keen.io tracking backends. It does not work with Google Analytics.

See this Wiki page for more information

Goals

Goals are uniquely tracked by default. i.e. if a goal is set to measure how many visitors clicked on a button, multiple clicks won't generate another goal completion. Only one per visitor. Non-unique goals can be set by passing

unique: false
to the goal when creating it.

Goals will only be tracked if the experiment was launched and a variant selected before. Tracking goals is therefore safe and idempotent (unless unique is false).

Here's a short sample of tracking multiple goals over multiple experiments:

import AlephBet from "alephbet";

// main goal - button click const button_click_goal = new AlephBet.Goal('button click'); $('#my-btn').on('click', function() { button_clicked_goal.complete(); });

// engagement - any click on the page const engagement = new AlephBet.Goal('engagement'); $('html').on('click', function() { engagement.complete(); });

const all_goals = [button_click_goal, engagement];

// experiments const button_color_experiment = new AlephBet.Experiment({ /* ... / }); const buy_button_cta_experiment = new AlephBet.Experiment({ / ... */ });

// adding all goals to experiments _(all_goals).each(function (goal) { button_color_experiment.add_goal(goal); buy_button_cta_experiment.add_goal(goal); });

// alternatively, you can use the add_goals method and pass it an array of goals button_color_experiment.add_goals(all_goals); buy_button_cta_experiment.add_goals(all_goals);

Custom Tracking Adapter

AlephBet comes with a built-in Google Analytics adapter and three, currently experimental, adapters with potentially better accuracy:

Persistent Queue GA Adapter

Persistent Queue Keen Adapter

Gimel adapter

Lamed adapter

Creating custom adapters is however very easy.

Here's an example for integrating an adapter for keen.io

(For a more complete implementation, you should use the built-in

Alephbet.PersistentQueueKeenAdapter
)

Custom Storage Adapter

Similar to the tracking adapter, you can customize the storage adapter. AlephBet uses localStorage by default, but if you want to use cookies or customize how data is persisted on the client, creating an adapter is very easy.

Here's a simple example of a cookie storage adapter with expiry of 30 days, using js-cookie:

 

Debug mode

To set more verbose logging to the browser console, use

import Alephbet from "alephbet"

AlephBet.options.debug = true`

Other install options

  • download
    alephbet.min.js
    from the
    dist
    folder on github
  • npm install alephbet
  • bower install alephbet

Analyzing results

AlephBet is built for developers, so there's no fancy interface or WYSIWYG editors. The best way to analyze the results and determine the best variant from an experiment is to look at the raw data and calculate the statistical significance. A couple of recommended resources:

Development

Commands

  • yarn run build
    - to build distribution files
  • yarn run watch
    - will watch files and re-build using
    jest

License

AlephBet is distributed under the MIT license. All 3rd party libraries and components are distributed under their respective license terms.

Copyright (C) 2015 Yoav Aner

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.