As you test the effectiveness of email marketing and develop your email marketing plan, there are several items you should routinely test. With about 90% of email opened in the 72 hours after it’s sent, you’ll have results of your tests in just three days.
Based on findings from an analysis of hundreds of thousands of email marketing promotions, as detailed in What’s Really Working in Email Marketing, here are the most important elements you should test:
Most likely, you’ll want to prioritize test elements and test them in phases. There will always be something for you to test. If you’re not testing the effectiveness of email marketing, you’re not going to improve. If you email large volumes, it’s advisable that you test 10% of your quantity first, and then roll out the winning email promotion and combination of elements to your remaining email addresses. To ensure a statistically reliable conclusion, a minimum of 10% of your email marketing volume should be allocated to testing.
Even if your email list is modest in size, you can still test. It’s advisable to have 5,000 email addresses in each test cell, but if you don’t have that volume, you can still divide your database into two groups and test two approaches. Even at small quantities you can get an idea of what will work better for you.
If you have 10,000 email addresses, chances are you’ll split these into two equal groups. If you have 20,000 or more email addresses, you might want to skip forward to Multiple Element Testing.
For illustration purposes, let’s assume you wish to test these multiple attributes with these subject lines and day of delivery:
Using these examples, if you have 10,000 or fewer email addresses, you would probably choose to test the day of delivery.
Why the day of delivery versus the subject line?
When testing the day of delivery, you will learn something you can apply to your next email promotion without appearing like you are spamming.
While you might email the winning subject line to the other half of your names that didn’t see it in your test, you risk alienating people on your list by mailing the same message (but with a different subject line) only a few days later. Subject line testing is important, but there may be more important attributes you test first.
Test Tuesday versus Thursday to see if there is a difference in response. If the response is markedly better for one day (let’s say it’s Tuesday), then you know that the next week you should test Tuesday against another day—let’s say Wednesday. If this test reveals that Wednesday is markedly stronger, then Wednesday is your new “control” and you should test still another day of the week against Wednesday. Keep testing as often as every three days until you pin down one day or another works better for you.
Note: in the preceding paragraph, the words “markedly stronger” were used as your comparison basis. If your day of delivery test doesn’t seem to reveal a day stronger than other days, pick the day that seems best to you and stick with it. Then you can move to testing the time of day following the same pattern described above. Or maybe you’ll decide to test pricing, discount amounts, or some other variable.
If your email list is somewhat larger (let’s say from 15,000 up to about 20,000 emails), you have the luxury of testing more variables at one time.
If you have 15,000 email addresses, then you should be able to split these into three groups and test three days of the week for delivery (for example, Wednesday versus Thursday versus Friday).
Multiple Element Testing
If you have 20,000 or more email addresses, you can start with a test plan that will segment names in four groups, and enable you to test two elements such as subject lines compared to day of delivery and have results in three days.
Your four test cells could be as follows:
Send these emails out on the day specified, but make sure you send all emails at the same time of day (you don’t want to introduce more than one variable per test, or your results won’t be attributable and will be unreliable).
If you email your first tests on Tuesday, then the remainder on Thursday, by Sunday you’ll have your results. Of the four test cells, one of them is bound to have worked best by a significant margin. Then you can do one of two things:
If for some reason you don’t see a significant margin of difference in those test results, test something else. There will always be a better approach somewhere that will improve your results.
An Expanded Test
If your email list is 40,000 or greater you can test three variables and get answers even faster.
We’ll assume that in addition to testing subject line and day of delivery that you want to test two different landing pages, one that is “Benefit-oriented” and another that is “Offer-driven.”
Now you’ll need to segment your list into eight cells. This test matrix looks like this:
Of course, you can continue to expand your email marketing test system if you have enough email addresses to support your efforts. It’s essential that you test the effectiveness of email marketing and develop your email marketing plan. When you test this approach, in 3 days you’ll have results that will be eye-opening and lead into what to test next so you can cash in on your email marketing effectiveness.