With A/B split testing, you easily test variants of an emailing and compare the results of the regular KPIs such as clicks or opens.
Ideas for testing are changes in subject lines, number of articles, styling, sending date or time or even sender details. Remember to test a single change at a time to make sure your test results are valid.
There are two ways an A/B split test can be started:
- For a regular scheduled email campaign: directly on a draft campaign with the scissor button
- In a marketing automation (MA) flow for trigger mails
Method 1: In Email Campaigns
Complete your base version of your mailing first. Since the variant is created as a copy, this will avoid having to edit multiple versions simultaneously. When it’s finished you can create a variant which contains the change you want to test. Second click the scissor button to create up to 10 variants in the “Split assistant” popup:
Here you also select whether you want a symmetrical (even) split or asymmetrical (uneven) split to present the variants to your contacts.
A symmetrical split will allow you to choose the winning variant. An asymmetrical split will send the variants to a selected sample number of contacts and determine the winning variant that will be sent to the rest of the target group.
You can have Maileon calculate the number of selected contacts and define the winning variant after a set period of time. After this period of time Maileon will automatically determine the winning variant and send out this variant to the remaining contacts. Tip: make sure to set your asymmetrical split test to last at least 24 hours, to make sure your contacts have enough time to open and click.
You can determine the number of sample contacts either with an absolute value, a relative percentage or calculate a statiscally recommended number of contacts:
To get the actual number of contacts, make sure to check the box to update your target group before calculating.
When the variants have been created they will be grouped in the overview:
The settings and content of the original version have been copied, but in the target group the difference is displayed marking the split in contacts.
At this point you make the required changes in the test. You can use the internal naming of the variants to reference the changes you’re testing. You’ll then need to plan sending for the variants separately. Make sure to use the same sending date/time unless you’re testing out different sending times.
Method 2: In Marketing Automations
The second testing method is to setup A/B split testing in an MA program. This can be done either as a standalone test or as a part of an existing MA program. Using the Split Test Branch Element you can setup the split test by configuring the (number of) variants and the duration or split logic.
The example below shows an example to test two variants. The winning variant can be determined either after a set amount of time or a fixed number of contacts entering the MA program, whichever comes first. In the example program below:
- Start by filter
- Split for a/b test to triggermails
- Duration 1 day, after that the winner takes all
