A/B testing lets you try two versions of a campaign on part of your list, then automatically send the better-performing version to everyone else. It's the fastest way to learn which subject line or message actually moves the needle — without guessing.
Set it up
- 1Write your main email on the Template tab — this is Variant A
- 2Open the Settings tab and turn on 'A/B Test'
- 3Enter Variant B — a different subject line, a different body, or both
- 4Choose what percentage of your list to test on
- 5Pick the winning metric: open rate, click rate, or reply rate
- 6Set how long to wait before picking the winner, then launch
How the test runs
- ▸CMass splits your test group evenly between Variant A and Variant B
- ▸It tracks opens, clicks, and replies separately for each variant
- ▸After your wait window, it picks the winner by your chosen metric
- ▸The winning variant is sent automatically to the remaining recipients
💡 Set the test size to 100% for a pure split test — both variants go to your whole list and no automatic winner is sent. Use a smaller percentage (say 30–50%) when you want CMass to auto-send the winner to the rest.
Reading the results
Open the Stats tab and expand the campaign. Each variant shows its sent count and open, click, and reply rates side by side, and the winner is marked with a trophy once it's been chosen and deployed.
✅ Test one thing at a time: For the clearest read, change only the subject line between A and B. If you change everything, you'll know which variant won — but not why.