Comments for Blog - Myna http://blog.mynaweb.com Tue, 21 May 2013 08:07:56 +0000 hourly 1 http://wordpress.org/?v=3.4.2 Comment on Myna versus A/B testing by Noel http://blog.mynaweb.com/2011/09/13/myna-versus-ab-testing/#comment-1635 Noel Tue, 21 May 2013 08:07:56 +0000 http://blog.mynaweb.com/?p=11#comment-1635 The experiments are comparing the number of conversions made from a fixed number of visitors. The 33% improvement is seen with 42'000 visitors, so what that means is we had 33% more conversions in 42'000 visitor due to displaying the better variant more often. There are two ways one can improve over stock A/B testing: by displaying the better more variant more often while a test is running, and by finishing a test earlier. These are really different ways of looking at the same thing, which is using the information you obtain while the test is running to make better decisions. The trick is doing it in a statistically sound way. The experiments are comparing the number of conversions made from a fixed number of visitors. The 33% improvement is seen with 42’000 visitors, so what that means is we had 33% more conversions in 42’000 visitor due to displaying the better variant more often.

There are two ways one can improve over stock A/B testing: by displaying the better more variant more often while a test is running, and by finishing a test earlier. These are really different ways of looking at the same thing, which is using the information you obtain while the test is running to make better decisions. The trick is doing it in a statistically sound way.

]]> Comment on Myna versus A/B testing by James J http://blog.mynaweb.com/2011/09/13/myna-versus-ab-testing/#comment-1633 James J Thu, 09 May 2013 04:55:53 +0000 http://blog.mynaweb.com/?p=11#comment-1633 Hi Noel, I'm not sure if I understand what you mean by "Myna has a whopping 33% improvement over A/B testing". Are you saying that Myna gets you there 33% quicker? I'm not sure if a statistical technique can improve the result of A or B. It can just get you to a result quicker. Maybe I'm too dumb to get this stats stuff. Please do explain... JJ Hi Noel,

I’m not sure if I understand what you mean by “Myna has a whopping 33% improvement over A/B testing”.

Are you saying that Myna gets you there 33% quicker?

I’m not sure if a statistical technique can improve the result of A or B. It can just get you to a result quicker.

Maybe I’m too dumb to get this stats stuff.

Please do explain…

JJ

]]>