Anatomy of Email List Cleaning | A Complete What,Why, How and Where

Author : Sandeep Saxena

 What is email list cleaning or email verification?

 In email marketing, email list cleaning stands for a process where we verify the validity of an email address before sending any actual email to it.

 There are different terms which are used in the industry for this verification process. Some of the most common terms are email verification, email cleaning, email hygiene, email scrubbing and email validation.  While some might argue that there are differences among these terms but more or less all of them are the same.

 Study shows that any email database degrades naturally by about 2% every month as people change or abandon their email boxes. When it comes to good emailing practices, email hygiene is a very important aspect. It helps email marketers in accomplishing better open and click rates, reduced bounces and reduced cost (because you are not sending tons of emails to invalid email addresses). 

 The email hygiene is a two step approach:

 Step 1 (Process or Practise part): The first step is more of a process where you develop certain good practices in your email marketing. You keep a close eye on your email lists and continuously remove or move around unengaged recipients. 

 For example you may want to develop a practise of removing (or moving) everyone from your main email list who has not opened any of your emails in the last 90 or 120 days. This is important because when you keep emailing to someone who has not opened any of your emails in the last 90 days it impacts the reputation of your domain and IPs and in turn overall deliverability. The person in fact may not have operated his mailbox for these days or may have simply abandoned that mailbox by now and in that case impact may be more severe. 

This first step is more of a good process or practise part and some tools may help you in achieving it by providing strong list segmenting features.

 You can try to re-engage the subscribers which you have moved away from the main list using a different IP and sub-domain. But before doing that it is important to verify these email addresses.

 Step 2 (Tools and Technology part): This step is more of a technical step where you actually use a tool to validate each email address.

 There are many reasons why you would do this. One reason can be that you have moved around your unengaged recipients as explained in step 1 and you want to reengage them.

 Another reason can be that you plan to reengage your very old subscribers whom you have not sent any emails for many months or years – you should check each and every email address on that list and remove all those which bounce or have become invalid. 

 In case your opt-in process is not foolproof there are chances that some spam traps may have sneaked in to your lists and in that case it is of utmost importance to clean your lists else you stand a high chance of getting your domains and IPs blacklisted. 

Let’s understand what happens exactly when we clean our email list.

 When we verify our email list, the tool used runs several checks on each email address. Some tools/services may also cross check the email address against a huge database they may have developed over many years to find out various details of the email address. 

 Some of the checks which are run by these tools are :

  1. Syntax verification: This step verifies for any syntax errors. For example an email address may be mistyped as [email protected]#com (The person has put a # in place of .)
  2. Duplicates: If you have merged many lists, there may be a chance that you have duplicates in your email list. Though most of the email marketing tools nowadays highlight or remove all duplicates when we export the lists – it’s better to remove them in earlier stages.
  3. Disposable Email Verification: There are some services which provide temporary and disposable email addresses which you can use with or without login for few minutes and then they become invalid. Its better to remove all these disposable email addresses from your list as these kind of email addresses are only used to check around the things on a temporary basis and in most probability it would bounce if you send them email later. 
  4. Spam Trap Detection: Spam traps are email addresses which are not human operated. They may be hidden on web pages in invisible form and if someone just scraps the pages for email addresses, the spam traps may sneak into his email lists. This may happen very easily if you have ever bought or borrowed an email list from someone. 
  5. Domain Verification or MX Record Verification: Verification of the MX servers for the domain in email address. This is to verify if its at all possible for that domain to receive emails.
  6. Mailbox Quota Verification: Verifies the mailbox to ensure if it can accept new emails or is it short of storage.
  7. Role Account Verification: In this step we remove all role based email addresses which are not operated by a single person but the whole team. Email addresses like [email protected], [email protected], [email protected], [email protected] all fall in this category. 
  8. Free Account Verification: Checks whether the email addresses are using free mailbox services like gmail.com and hotmail.com
  9. Catch-all: Catch-all are domains which just accept any email address sent to them even if the mailbox does not exist. These domains are categorised separately as it’s quite risky to send emails to these domains unless we have an opted-in engaged recipient.
  10. Bounces: In this step tools use SMTP  protocol to find out whether the email can be delivered or would it bounce back. Usually it is done by opening an SMTP connection with the receiving server.
  11. Deliverability Score: Some services also give a score to each email address based on the historical data they have. 

A good email verification tool would take care of all these validations and prepare a report highlighting all of these categories. The terminology may differ for different tools – for example some tools might club everything which is not deliverable under “Undeliverables” while others highlight them under different categories like “Roles”, “Temporary mailbox”, “Catch-all”, etc. 

 Let’s see what tools are available to clean email lists. 

 There are a number of SAAS based email cleaning tools available online. You just need to put “email cleaning” in Google and you would see plenty of them. They charge on the basis of the number of email volumes you need to clean. Most of these SAAS tools are very effective. They provide a nice dashboard where you can upload your data and in a short time they clean the list and generate a report which shows the status of each email address.

 Apart from these online tools, there are few other options available which you can explore and determine if they fit your purpose. 

 Let’s see what all tools are available for list cleaning:

 1. Open-source: There are some open source tools available on GitHub. These are written in different scripting or programming languages. Installation instructions are available on their GitHub pages. 

 Some of these are :

TrueMail: https://github.com/rubygarage/truemail

TrueMail on GitHub

Email List Cleanerhttps://github.com/subimage/email_list_cleaner

SMTP Validate Emailhttps://github.com/zytzagoo/smtp-validate-email

Trumailhttps://github.com/trumail/trumail

You would need a very good developer or a team of developers to effectively deploy these open source. It may take some time to deploy it and then another significant amount of time before you can use it at a large scale. 

 If you have regular requirements of very high volume list cleaning then you might want to explore this option (at your own risk 😄 😄 😄). But if one of these  deployed successfully and scaled to make large list cleaning possible then you can save big money and who knows you may want to start your own list cleaning company in future.

2. Free Online tools: Some individual developers have developed their own list cleaning tools and made them available online. These tools are free to use. These tools may or may not cover all of 11 cleaning aspects explained earlier. But in case you just need to remove roles, duplicates, invalid MX, etc you can use them.

 There is a free online tool at : https://leopathu.com/verify-email

 We at Postbox Consultancy Services also have a free verification service at https://emailverify.postboxservices.com. Our tool is for very basic verification and it does not remove bounces or spam traps.  As said earlier it’s a simple deployment in case you need to remove duplicates, invalid syntax, roles, etc and it is not at all meant for a professional level cleaning.

 

Free Email List Cleaning Service from Postbox Consultancy Services
Free Email List Cleaning Service from Postbox Consultancy Services

 3. Licensed self-hosted applications: Third in line are the licensed applications which you can deploy in-house.

 We recently found a solution BounceKick – a self hosted email verification software. Bouncekick is an elegant and fast email verifier application which can be installed on your own infrastructure. You just need to procure the license once and after that there is no monthly subscription or payment.

 

Postbox Services Email List Cleaning
BounceKick Dashboard

It has an inbuilt REST API to integrate with any third-party application to verify the emails on the fly. 

BounceKick API
BounceKick API Example
BounceKick Features:-

—————————-

  • Do unlimited email verification for a lifetime with a one time fee
  • Keep your email database secure as it resides in your own datacenter
  • BounceKick verifies up to 100k emails per day (Based on the SMTP server acknowledgement)
  • The upload CSV file can have up to 100k emails per list
  • BounceKick verifies the emails via SMTP servers to get accurate results
  • To avoid spam traps and IP Block it uses the reverse DNS algorithm
 If you are interested to get BounceKick self hosted email verification system deployed and avail a discount, please leave us your email address using this form and we would get back to you. 

 4. SAAS based services: There are many SAAS based online list cleaning tools available.  These are very effective and can remove most of the bounces, catch-all and some spam traps. 

 These online services charge on the basis of the email volume you need to clean. Most of them provide API integrations which can be called from your application for real time email verification.  Some of them also provide ready made integrations/plugins with popular CRMs and email marketing applications. 

 One of such great services is Bouncer. You can check this blog about Bouncer and sign-up with them in case you need a quick, professional and cost effective list cleaning.

Bouncer Dashboard
Bouncer Dashboard

 Many other services are listed in this article from martech.zone : https://martech.zone/verify-email-lists

In future we will review some of these services so stay tuned.

 5. Verification tools inbuilt with email marketing software: Last in line are the tools which are inbuilt in various email marketing applications. They are provided as an extra paid feature with some email marketing applications. Some applications provide verification feature free or as extra credits on the basis of the email marketing plan you purchase. 

 The goal here is to give you a facility to clean the list so that the platform maintains its IP reputation (in case emails go out from the platform’s own IP addresses) and you get good open rates.

One example here is Snov.io which has inbuilt email verification on their dashboard. 

Snov.io Email Verification
Snov.io email verification feature

If you need any help or consultation with list cleaning please don’t hesitate to contact us

Contact Us

Sandeep Saxena
Author:
Sandeep Saxena is CEO at Postbox Consultancy Services. He is working as an email marketing and deliverability consultant for last 5 years. Before venturing in to Postbox Consultancy Services, Sandeep worked in to IT industry for close to 10 years as a DevOps consultant. Sandeep is based in Bhopal, India and when not working he is often seen reading a book or doing meditation.

Leave a Reply

Your email address will not be published. Required fields are marked *