Anatomy of Email List Cleaning | A Complete What,Why, How and Where
Author : Sandeep Saxena
What is email list cleaning or email verification?
In email marketing, email list cleaning stands for a process where we verify the validity of an email address before sending any actual email to it.
There are different terms which are used in the industry for this verification process. Some of the most common terms are email verification, email cleaning, email hygiene, email scrubbing and email validation. While some might argue that there are differences among these terms but more or less all of them are the same.
Study shows that any email database degrades naturally by about 2% every month as people change or abandon their email boxes. When it comes to good emailing practices, email hygiene is a very important aspect. It helps email marketers in accomplishing better open and click rates, reduced bounces and reduced cost (because you are not sending tons of emails to invalid email addresses).
The email hygiene is a two step approach:
Step 1 (Process or Practise part): The first step is more of a process where you develop certain good practices in your email marketing. You keep a close eye on your email lists and continuously remove or move around unengaged recipients.
For example you may want to develop a practise of removing (or moving) everyone from your main email list who has not opened any of your emails in the last 90 or 120 days. This is important because when you keep emailing to someone who has not opened any of your emails in the last 90 days it impacts the reputation of your domain and IPs and in turn overall deliverability. The person in fact may not have operated his mailbox for these days or may have simply abandoned that mailbox by now and in that case impact may be more severe.
This first step is more of a good process or practise part and some tools may help you in achieving it by providing strong list segmenting features.
You can try to re-engage the subscribers which you have moved away from the main list using a different IP and sub-domain. But before doing that it is important to verify these email addresses.
Step 2 (Tools and Technology part): This step is more of a technical step where you actually use a tool to validate each email address.
There are many reasons why you would do this. One reason can be that you have moved around your unengaged recipients as explained in step 1 and you want to reengage them.
Another reason can be that you plan to reengage your very old subscribers whom you have not sent any emails for many months or years – you should check each and every email address on that list and remove all those which bounce or have become invalid.
In case your opt-in process is not foolproof there are chances that some spam traps may have sneaked in to your lists and in that case it is of utmost importance to clean your lists else you stand a high chance of getting your domains and IPs blacklisted.
Let’s understand what happens exactly when we clean our email list.
When we verify our email list, the tool used runs several checks on each email address. Some tools/services may also cross check the email address against a huge database they may have developed over many years to find out various details of the email address.
Some of the checks which are run by these tools are :
- Syntax verification: This step verifies for any syntax errors. For example an email address may be mistyped as [email protected]#com (The person has put a # in place of .)
- Duplicates: If you have merged many lists, there may be a chance that you have duplicates in your email list. Though most of the email marketing tools nowadays highlight or remove all duplicates when we export the lists – it’s better to remove them in earlier stages.
- Disposable Email Verification: There are some services which provide temporary and disposable email addresses which you can use with or without login for few minutes and then they become invalid. Its better to remove all these disposable email addresses from your list as these kind of email addresses are only used to check around the things on a temporary basis and in most probability it would bounce if you send them email later.
- Spam Trap Detection: Spam traps are email addresses which are not human operated. They may be hidden on web pages in invisible form and if someone just scraps the pages for email addresses, the spam traps may sneak into his email lists. This may happen very easily if you have ever bought or borrowed an email list from someone.
- Domain Verification or MX Record Verification: Verification of the MX servers for the domain in email address. This is to verify if its at all possible for that domain to receive emails.
- Mailbox Quota Verification: Verifies the mailbox to ensure if it can accept new emails or is it short of storage.
- Role Account Verification: In this step we remove all role based email addresses which are not operated by a single person but the whole team. Email addresses like [email protected], [email protected], [email protected], [email protected] all fall in this category.
- Free Account Verification: Checks whether the email addresses are using free mailbox services like gmail.com and hotmail.com
- Catch-all: Catch-all are domains which just accept any email address sent to them even if the mailbox does not exist. These domains are categorised separately as it’s quite risky to send emails to these domains unless we have an opted-in engaged recipient.
- Bounces: In this step tools use SMTP protocol to find out whether the email can be delivered or would it bounce back. Usually it is done by opening an SMTP connection with the receiving server.
- Deliverability Score: Some services also give a score to each email address based on the historical data they have.
A good email verification tool would take care of all these validations and prepare a report highlighting all of these categories. The terminology may differ for different tools – for example some tools might club everything which is not deliverable under “Undeliverables” while others highlight them under different categories like “Roles”, “Temporary mailbox”, “Catch-all”, etc.
Let’s see what tools are available to clean email lists.
There are a number of SAAS based email cleaning tools available online. You just need to put “email cleaning” in Google and you would see plenty of them. They charge on the basis of the number of email volumes you need to clean. Most of these SAAS tools are very effective. They provide a nice dashboard where you can upload your data and in a short time they clean the list and generate a report which shows the status of each email address.
Apart from these online tools, there are few other options available which you can explore and determine if they fit your purpose.
Let’s see what all tools are available for list cleaning:
1. Open-source: There are some open source tools available on GitHub. These are written in different scripting or programming languages. Installation instructions are available on their GitHub pages.
Some of these are :
Email List Cleaner: https://github.com/subimage/email_list_cleaner
SMTP Validate Email: https://github.com/zytzagoo/smtp-validate-email
You would need a very good developer or a team of developers to effectively deploy these open source. It may take some time to deploy it and then another significant amount of time before you can use it at a large scale.
If you have regular requirements of very high volume list cleaning then you might want to explore this option (at your own risk 😄 😄 😄). But if one of these deployed successfully and scaled to make large list cleaning possible then you can save big money and who knows you may want to start your own list cleaning company in future.
2. Free Online tools: Some individual developers have developed their own list cleaning tools and made them available online. These tools are free to use. These tools may or may not cover all of 11 cleaning aspects explained earlier. But in case you just need to remove roles, duplicates, invalid MX, etc you can use them.
There is a free online tool at : https://leopathu.com/verify-email
We at Postbox Consultancy Services also have a free verification service at https://emailverify.postboxservices.com. Our tool is for very basic verification and it does not remove bounces or spam traps. As said earlier it’s a simple deployment in case you need to remove duplicates, invalid syntax, roles, etc and it is not at all meant for a professional level cleaning.
3. Licensed self-hosted applications: Third in line are the licensed applications which you can deploy in-house.
We recently found a solution BounceKick – a self hosted email verification software. Bouncekick is an elegant and fast email verifier application which can be installed on your own infrastructure. You just need to procure the license once and after that there is no monthly subscription or payment.
It has an inbuilt REST API to integrate with any third-party application to verify the emails on the fly.
- Do unlimited email verification for a lifetime with a one time fee
- Keep your email database secure as it resides in your own datacenter
- BounceKick verifies up to 100k emails per day (Based on the SMTP server acknowledgement)
- The upload CSV file can have up to 100k emails per list
- BounceKick verifies the emails via SMTP servers to get accurate results
- To avoid spam traps and IP Block it uses the reverse DNS algorithm
4. SAAS based services: There are many SAAS based online list cleaning tools available. These are very effective and can remove most of the bounces, catch-all and some spam traps.
These online services charge on the basis of the email volume you need to clean. Most of them provide API integrations which can be called from your application for real time email verification. Some of them also provide ready made integrations/plugins with popular CRMs and email marketing applications.
Many other services are listed in this article from martech.zone : https://martech.zone/verify-email-lists.
In future we will review some of these services so stay tuned.
5. Verification tools inbuilt with email marketing software: Last in line are the tools which are inbuilt in various email marketing applications. They are provided as an extra paid feature with some email marketing applications. Some applications provide verification feature free or as extra credits on the basis of the email marketing plan you purchase.
The goal here is to give you a facility to clean the list so that the platform maintains its IP reputation (in case emails go out from the platform’s own IP addresses) and you get good open rates.
One example here is Snov.io which has inbuilt email verification on their dashboard.
Hope you enjoyed reading the blog. In case of any queries please feel free to approach us.
If you need any help or consultation with list cleaning please don’t hesitate to contact us