A/B Testing Example

A/B Testing Example

I had a chance to speak at the Tampa Bay Tech Forum last night at The Wave in downtown Tampa. Thanks to Heather Kenyon, Kimberly Wander and the team at TBTF,  as well as Mike O’Donnel for invite and the attendees for a fun event.

One of the items I talked about was how to do A/B testing for startups. Specifically using a tool like HelloBar to start your A/B testing in 5 minutes with minimal tech skills. A/B testing is the process of using two variants to validate your hypothesis, this testing can be done for pricing as well as messaging, but you need to test one item at a time. The winner of the test becomes “A” and you start the process over again with a new “B” test as the test variable.

While on the plane today flying home today, there was a great example of the how larger/non startup companies use A/B Testing. I had my phone out to login to gogo to look at the price on my short Delta flight leaving from Tampa to Atlanta, then on to the long flight from Atlanta to Seattle. When I logged in and saw there was a $9.95 day rate.

When I was finally able to get to my laptop, I connected my to gogo and was greeted by the following (less)welcome screen. The offer, now $14.95 per day.

So gogo is either testing pricing based on use type. Or there is a $5 bug in their login process. I saved the $5 and registered with my mobile, logging into the same account with my laptop after.

In either case, I’m sure gogo is tracking the number of users that go from welcome page to paid using both mobile and laptops.

2 Replies to “A/B Testing Example”

  1. I ave had the sae experience with GoGo. My belief is they were using the device type as a surrogate for bandwidth utilization in their pricing model, knowing a laptop uses more bandwidth than a mobile device. (Not as ‘chatty;’ doesn’t have a bunch of apps running in the background also using bandwidth; loads smaller, mobile web pages, etc.) you found the same work-around I have used. Apparently their application doesn’t check device type when ou log in. Nice for us…

Leave a Reply

Your email address will not be published. Required fields are marked *