Gab Goldenberg says:
Hi Gab – thanks for the comment.
In this case, the objective was to challenge the best practice “Always place the CTA above the fold.” The research question was: Which variant will before better – the control /CTA above the fold) or the treatment (CTA below the fold).
The test design was a simple A/B split with a 50/50 distribution between variants. The sample size was 100 conversions (conversions not visits), the statistical confidence level was 98%, and the standard error was <1%.
The result was that the treatment outperformed the control by 304%, and the answer to the research question was “The treatment (CTA below the fold) performed better than the control (CTA above the fold).”
I’d venture to that all the abovementioned facts provide the basis for a pretty conclusive test…
Testing a variant 30-day trial on a short-form landing page is interesting. However, that would be a completely different test and we’d no longer be challenging the “Above the fold” myth, we’d be testing whether a short-form LP performs better than a long-form LP, or whether a free trial performs better than an introduction offer. Thus we would also have to come up with a different research question.
But funny you should mention it because the first thing I did for this client was to challenge their original short-form LP that consisted of a headline, a few bullets, and a paragraph. My long-form LP more than doubled conversions…
A free 30-day trial is an interesting approach but – as mentioned in the post – the product in question is a physical service where you get ingredients and recipes delivered to your door. The costs involved in the supply chain and delivering the product make it difficult to justify a free 30-day trial. Thus the company has chosen to go with a special introduction offer involving a discount.
Thanks for the heads up on the caps, it’s setup by default in the theme I’m using, but I’ll look into changing it.