All postsJudging the effectiveness of hosted email protection isn’t as cut and dry as it might seem. Email security services don’t just block or allow email. In between is a host of options. Choices that walk a tightrope, where one false move can cost a business lost revenue. Either because malware found a way in, or a genuine message kept out.
Threats That Outsmart People and Technology
Email is still one of the most frequently used attack vectors to gain access into an organisation’s network. The “ILOVEYOU” virus might have been twenty-five years ago, but the human element is just as easy to exploit.
The latest Business Email Compromise (BEC) attacks are extremely lucrative for successful hackers. By exploiting trust and authority, these attacks attempt to bypass traditional security mechanisms by deliberately omitting malicious links or attachments.
Instead, they rely on social engineering tactics to fool the user with extremely convincing fraudulent emails that appear to come from legitimate sources. These seemingly genuine messages often trick recipients into transferring money, sharing sensitive information or performing other actions that can compromise a business.
This tactic not only makes them incredibly dangerous but also quite hard to spot by either people, or technology.
Malicious Email Detected: Now What?
While reporting detection rates is relatively straight-forward, it’s an unsubtle way to compare email security services, because not all providers handle threats in the same way.
A service might completely delete an incoming malicious email and never allow the intended recipient to see it – thereby removing any possibility of accidental interaction. If a service lacks the utter conviction that the message is unwanted, it might condemn the suspicious message to a quarantine area.
While this keeps the immediate threat away from recipients, it still potentially leaves it up to the user to decide whether or not the message is safe. But sometimes security personnel need to see what’s coming in, so quarantines can be useful investigation tools. In our testing, we make a distinction between whether a quarantine is user-based or admin-only.
At the weaker end of the scale, a service might simply add a warning to the email’s Subject line. But at SE Labs, our view is that keeping threats as far away from the user as possible is best.
During a test, we measure everything and attribute points that form the final ratings depending on the action taken by the email security service. For example, a service that completely blocks a malicious message from falling into the hands of its intended recipient is rated more highly than one that prefixes the Subject line with “Malware:” or “Phishing attempt:” or sends the message to a ‘Junk’ folder.
When Good Emails Get Flagged: The False Positive Problem
At the same time, email security service providers need to ensure that all legitimate messages arrive in the inbox. It would be easy to create a product that blocked all threats if it was also allowed to block all legitimate email.
Categorising how a service handles genuine messages is similar to how it administers threats, but in reverse. Making a small change to the Subject line is much less serious an error than deleting the message and failing to notify the recipient. If a legitimate email is categorised by a service as a threat during our testing, then a ‘false positive’ result is recorded, and points are taken away.
It’s important to test for false positives because too many indicates a product that is too aggressive and will block useful email as well as threats. Finding the balance between allowing good and blocking bad is the key to almost every type of security system.
Pushing Email Security Further, One Test at a Time
Our ratings in judging the performance of email security services reflect the relative importance we assign to each outcome. However, recognising that not all organisations have the same view, and because of the transparent way we implement tests, it’s also possible for businesses to take the raw data from our reports to roll out their own set of personalised ratings.
To date, no vendor has scored 100% Total Accuracy in our Advanced Email Security Service tests. With so many different moving parts in judging if an email is malicious, versus the danger of a customer losing revenue due to miscategorising legitimate messages, it’s extremely difficult.
Many email security services have failed to pass the grade, and very few have been awarded SE Labs’ coveted AAA Award for Advanced Email Security. This year so far, it’s only been awarded to two companies. You can read the full reports on our website: Cisco Secure Email Threat Defense and Coro Email and Cloud Security.
