We use Mixpanel as one of the main tools in our metrics arsenal. It’s great for capturing any event that we think is interesting, and for reporting on conversion funnels.
Since we’ve already defined the events that are important to our business, we can set up A/B tests for all sorts of aspects of our site and measure how they impact those conversions we are already watching.
Let’s say that we have a simple signup conversion defined. We’ll start with an event triggered when a visitor reaches our signup form. If they successfully submit the form a second event is triggered and we count them as having converted. Pretty standard.
Now we have some intro copy on the page asking the user to sign up, and we want to test how altering that language affects the rate at which people make the leap. We have the original paragraph as the control, and we come up with a few alternate options to try out. When we send the form event to Mixpanel, we include a property for this A/B test. We decided to use a unique identifier for the test as the key, and a label for the option the user saw as the value.
In Mixpanel we can drill down into our funnel to examine the breakdown for our test. Here’s what it looks like:
You can get a pretty good ballpark idea of how the different options are performing just by glancing at this chart. For more discipline we’ll want to use the API to pull these numbers out and run them through some stats equations you probably forgot in high school. Stay tuned for more on that process in a later post.