All postsHow SE Labs tests and scores email security services
Email security services can do a lot to protect users from online threats. This also means a lot can go wrong too. Testing and scoring these services requires a lot of attention to detail and a scoring method that takes into account all of the possible outcomes, including unexpected ones.
Learn:
- How email security services work
- How you should use them
- Why we score services the way we do
How do email security services work?
At their most basic, and email security service will allow legitimate emails into the inbox, and block malicious ones. By malicious we usually mean malware, phishing attacks and other types of social engineering.
There are many ways for an attacker to run an attack. For example, an attack involving malware will rarely involve attaching malicious code to an email. It is more common to make these files available using links to websites or file hosting services. An email security service might need to check web links in email messages, rather than just scan for bad attachments.
Security vendors and their customers don’t like mistakes. If legitimate email messages are lost, or bad ones allowed through, the service has failed. As a result, services sometimes ‘hedge’ on a decision. Instead of deleting a bad email it might send it to the Junk folder. Or to a quarantine area where someone can review it before the user downloads and opens it.
Handling threats
Here are the four most common way for email security services to handle suspected threats:
- Block – This can involve quietly deleting the message or rejecting it, sending a response to the sender.
- Quarantine – Placing an suspected email into an online folder keeps it away from a user’s inbox. If only an administrator can access that folder, the service protects the user from making a mistake and restoring a bad email.
- Edit – An email service can make changes to an email. It may remove attachments, check and remove malicious links or add warnings that the email is suspicious.
- Junk – Incoming threats may appear in a Junk folder in the user’s email client. Easily available for review, the risk is that users will make a poor decision and open a malicious message. At this stage the email service has put a lot of the decision-making into its customer.
Keep threats at arm’s length
We believe that the ideal way to handle threats is to keep them as far from users as possible. As such, blocking them is the most effective way to go. It is, however, the most risky… If a service makes a mistake and deletes a legitimate email, no-one knows that the email was lost. Rejecting an email with a warning to the sender at least let’s them know they failed to reach the recipient. This is fine, as long as the sender isn’t an attacker! If hackers receive warnings they can keep trying until they get through.
Adding warnings to emails gives users some confidence that the email could be bad, but they still need to decide for themselves. A service that claims to have removed malicious content gives the user further confidence, which is fine as long as it didn’t leave malicious content in place, along with a false sense of confidence.
When things go wrong
As mentioned, blocking a legitimate email is far from ideal, but things can actually be worse. Imagine that a threat arrives and the system analyses it. It contains both an attached document and a link to a file on Dropbox. The email security service removes the attached file but leaves the link intact. The user is still potentially exposed to the threat, should they click on the link.
If the service gives an alert that it has removed the threat, the user could now labour under a false sense of security. Maybe they would have been shy about clicking the link, but the service has detected and removed the threat so all seems to be well.
Spam folders are notoriously inaccurate. How many times have major companies asked you to check your spam folder when you first sign up? Criminals are good at tricking people – it’s their job. Some spam is so obvious that you ignore it, but relatively unsophisticated social engineering attempts can fool you. You remove the email from the spam folder and the service has failed you.
Quarantine features are probably one of the best compromises you can adopt, whether you are a business or home user. Ideally all suspect emails would go to a locked area accessible only by a skilled administrator. They can then release falsely imprisoned messages and leave the bad ones to rot.
User-accessible quarantine at least requires the user to log in and check, which is harder to do than scan through a junk folder. For a home user this is probably the best choice. Businesses should use admin-level quarantines.
Wanted: Handling legitimate email
Legitimate messages can face similar fates. Mistakes can hide wanted emails in junk or quarantine folders, or remove useful links and documents. In our email security testing we penalise services that make errors with legitimate messages using the same scale that we use for succeeding or failing with threats.
Different types of email security service
Usually security vendors provide email security services in one of two ways:
- External gateway
- Integrated with an email service
For example, a vendor might provide a service that monitors incoming email and sends acceptable message into your regular email service, such as Microsoft Office 365 or Google Workspace (aka ‘G Suite’). These ‘gateway’ services often involve setting up DNS MX records. They sit outside your business environment and have limited insight into emails sent within the organisation.
Integrated service are embedded into the email service. Microsoft and Google both provide in-built protection and Microsoft has an additional advanced protection module you can buy. You can also plug in services from other vendors. For example, Cisco has a service that integrates directly with Microsoft Office 365.
Many vendors provide both gateway and integrated services. There are pros and cons to each approach, including cost. If you don’t use a cloud service then you probably have to take the gateway approach. Gateway solutions are well-established but can be harder to configure and can’t monitor internal email traffic. Integrated solutions may be able to detect threats some time after they arrive.
Conclusion
Email security services do not just block and allow email. They make changes to messages, move mail to more or less accessible locations and can add warnings. The further a service keeps a threat from a user, the safer that user us. The fewer mistakes the service makes, with both threats and legitimate messages, the better. The email testing conducted by SE Labs takes all of these eventualities into account.
