The Goal
The primary goal was to convert more email subscribers into paid subscribers. We wanted to do that without losing current subscribers to aggressive emailing. So, we had to test strategies for SMS and
257%
increase in campaign revenue
175%
increase in adjusted flow revenue
63%
list growth
The Strategy
The first measure was to ensure we were sending regular campaigns so that we could eliminate fluctuations in the data from different sending intervals. This regular sending schedule was also going to be our main revenue driver for campaigns, before starting optimizations with strategic tests.
1
Once we had established a good sending frequency, we started testing send times. We set up different tests for weekdays and weekends. The results were not always statistically significant, but we established ideal send times after systematic testing for a few months.
2
We also tested different segmentation strategies, focusing on various signals (email activity, site activity, purchase behavior). Purchasers were one of our most important groups, as driving repeat purchases was the first step towards turning a one-off buyer into a subscriber. We ended up finding engagement levels that maximized revenue while keeping churn low.
3
Despite excluding current subscribers from our campaigns, we were dealing with very skewed reporting numbers as a number of flows was targeting current subscribers. We built reporting templates that cleaned the data, removing revenue that was not driven by Klaviyo and painting a much clearer picture of our email performance. The performance numbers you can see above and below are all based on that corrected revenue.
4
203%
increase in email revenue
257%
increase in campaigns revenue
175%
adjusted flow revenue
63%
list growth
-28%
unsubscribe rate
-41%
spam complaint rate
Summary
PlantPaper had a good email strategy in place, but they wanted to use data to understand exactly how many emails to send, at what time, and to which parts of their audience. Setting up proper conditions for reliable A/B-testing and creating accurate reporting allowed them to get a real picture of their email performance and improve it with targeted testing.
Lessons Learned
Create the right conditions
The hardest thing in marketing A/B-tests is making sure the test has the right conditions to work. Especially when testing over extended periods, we need to make sure to minimize any factors that could disrupt our test - and there are many of them. Thinking about this ahead of time and creating a plan helps a lot when executing these tests over time.
1
Test many different things
You don't know what will be successful before testing it. If the question is worth asking, then it's worth testing.
3
Accurate Reporting for Subscription Brands
Klaviyo will attribute a lot of subscription sales if you use it to send any messages to current subscribers. In order to understand what is really going on, it is extremely important to create a reporting framework that corrects for these attribution mistakes.
2
Sometimes tests don't work
Sometimes, we have to accept that a test is inconclusive. A lot of people rush to conclusions to avoid the feeling of having wasted time. This is a mistake, as it leads to strategies that are not based on a sound foundation. When a test is inconclusive, that simply means the tested object, or the 2 variations, are not as important as we thought. That, in itself, is a finding worth logging.
4
Grow your email marketing revenue with a data-driven approach
SCHEDULE A FREE STRATEGY SESSION