Why Mailchimp Open and click rates aren’t reliable
Quite understandably, it’s easy to think of open rates as a good way to understand how successful an email campaign was. Presumably if the reported open rate is high, that means lots of people read the email, right?
Wrong.
Back in the good old days, open rates were fairly reliable. Then people got mobile phones, Apple made more updates, and suddenly open rates stopped being useful.
The reason for this is that opens are actually reported by the download of a tiny 1x1 pixel image that Mailchimp (and the other email marketing platforms) add to the email as it is being sent out. Apple devices download that image alongside any others in the email as it arrives, so that there’s no lag when the email is opened by the actual human bean. So that means that the person might have opened it, or they might not have - either way, the Apple device will report back to Mailchimp (Mailerlite, Klayivo, Kit etc are all the same, this isn’t an issue that specifically affects Mailchimp), that the email was opened.
That means that your open rates are very likely to be much lower than the reported open rate state.
You can adjust the reports to remove the influence of “Apple MPP” to see a purer figure of the open rate of everyone who looked on a non-Apple device and the results can be quite sobering.
It also definitely affects certain audiences more than others - those who are more likely to be looking at your emails on their mobiles (perhaps because of who they are or the time of day you send them) and creative industries where they use more Macs.
And another factor to add into the mix is firewalls used by many organisations, which can really skew the information fed back to you, and also do bot clicks on your links before to make sure your emails are OK before letting them through. All of this impacts on the data in your Mailchimp reports.
All is not lost though
That doesn’t mean they’re completely meaningless - for example if you send an email regularly to a specific segment or your whole audience, the same manipulating factors will apply to each email. So comparing your own campaigns’ open rates against one another can be useful to get a feel for which perform better than others.
The measure of an email’s success isn’t ultimately about the opens or the clicks - it’s about the enquiries, the replies, the conversions, the sales, the bookings, the survey responses etc etc. That should be your number 1 metric, which can’t always be even seen in Mailchimp. After all - the reports simply can’t tell you if someone read your email then phoned you up with credit card in hand.
5 reasons this really matters
Aside from the general ability to understand your reports, this also has a significant bearing on the following:
Using whether someone opened an email as a factor in a segment - if Mailchimp, and therefore you, don’t accurately know if someone opened an email, it doesn’t make sense to use that as a reliable factor in segmentation.
Typically clicks are more reliable, but not always. Unless you’re doing B2B and have lots of larger companies with firewalls in your mailing list, click rates are usually a much better way to gauge engagement. But ultimately those measures of success that I talked about just up there 👆 are what counts.
Similarly, I get asked a lot about using if someone opened an open as a factor in an automation sequence e.g. if they opened an email, do x, if not, do y. But if we don’t actually know if they opened it, that doesn’t make sense to use.
Even if someone did actually open the email, they might have only read the first 2 lines, or it could have just been to scroll to the bottom and unsubscribe. An open doesn’t mean engagement. Although clearly someone can’t engage with it unless they open.
Opens are obviously good. But just because someone doesn’t open your emails, it doesn’t mean they won’t ultimately contact you - you simply showing up in their inbox once in a while reminds them of your existence for when they are ready to reach out.
I see a lot of email senders and email marketing experts who sell services and advertising space within their emails by merrily quoting how great their open rates are to demonstrate the value they add. We know open rates aren’t reliable, so be very sceptical.
Got questions? Need help?