Testing anti-breach products needs the full chain of attack. Symantec Endpoint Security Complete is the first endpoint detection and response offering to face our brand new Breach Response Test.
All posts
Tag: hacking
All posts
All posts
Targeted attacks with public tools
We run attacks with public tools to keep our tests accurate and useful.
Over the last few years we have tested more than 50 different products using over 5,000 targeted attacks. And there’s news, both good and bad.
In this article we will look at the different tools available, how effective they are at helping attackers bypass anti-malware products and how security vendors have been handling this type of threat for over a year.
All posts
Assessing next-generation protection
Malware scanning is not enough. You have to hack, too.
Latest report now online.
The amount of choice when trialling or buying endpoint security is at an all-time high. ‘Anti-virus’ first appeared 36 years ago and, in the last five years, the number of companies innovating and selling products designed to keep Windows systems secure has exploded.
And whereas once vendors of these products generally used non-technical terms to market their wares, now computer science is at the fore. No longer do security firms offer us ‘anti-virus’ or ‘hacker protection’ but artificial intelligence-based detection and response solutions. The choice has never been greater. Nor has the confusion among potential customers.
Assessing next-generation protection is not easy.
While marketing departments appear to have no doubt about the effectiveness of their product, the fact is that without in-depth testing no-one really knows whether or not an Endpoint Detection and Response (EDR) agent can do what it is intended.
Assessing next-generation protection
Internal testing is necessary but inherently biased: ‘we test against what we know’. We need through testing, including the full attack chains presented by threats. That’s how to show not only detection and protection rates, but response capabilities.
EventTracker asked SE Labs to conduct an independent test of its EDR agent, running the same tests as are used against some of the world’s most established endpoint security solutions available, as well as some of the newer ones.
This report shows EventTracker’s performance in this test. You can compare the results directly with the public SE Labs Enterprise Endpoint Protection (Oct – Dec 2018) report, available here.
All posts
How well do email security gateways protect against targeted attacks?
Email security gateways protection: Email security test explores how and when services detect and stop threats
Latest report now online.
This new email protection test shows a wide variation in the abilities of the services that we have assessed.
You might see the figures as being disappointing. Surely Microsoft Office 365 can’t be that bad? An eight per cent accuracy rating seems incredible.
Literally not credible. If it misses most threats then organisations relying on it for email security would be hacked to death (not literally).
Email security gateways protection
But our results are subtler than just reflecting detection rates and it’s worth understanding exactly what we’re testing here to get the most value from the data. We’re not testing these services with live streams of real emails, in which massive percentages of messages are legitimate or basic spam. Depending on who you talk to, around 50 per cent of all email is spam. We don’t test anti-spam at all, in fact, but just the small percentage of email that comprises targeted attacks.
In other words, these results show what can happen when attackers apply themselves to specific targets. They do not reflect a “day in the life” of an average user’s email inbox.
We have also included some ‘commodity’ email threats, though – the kind of generic phishing and social engineering attacks that affect everyone. All services ought to stop every one of these. Similarly, we included some clean emails to ensure that the services were not too aggressively configured. All services ought to allow all these through to the inbox.
So when you see results that appear to be surprising, remember that we’re testing some very specific types of attacks that happen in real life, but not in vast numbers comparable to spam or more general threats.
Threats at arm’s length
The way that services handle threats are varied and effective to greater or lesser degrees. To best reflect how useful their responses are, we have a rating system that accounts for their different approaches. Essentially, services that keep threats as far as possible from users will win more points than those who let the message appear in or near the inbox. Conversely, those that allow the most legitimate messages through to the inbox rate higher than those which block them without the possibility of recovery from a junk folder or quarantine.
If you spot a detail in this report that you don’t understand, or would like to discuss, please contact us via our Twitter or Facebook accounts.
SE Labs uses current threat intelligence to make our tests as realistic as possible. To learn more about how we test, how we define ‘threat intelligence’ and how we use it to improve our tests please visit our website and follow us on Twitter.
Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports.
All posts
Network security appliances vs. Word and PowerShell
Network security appliances tested. Over the last few months we have seen a surge in attacks using apparently innocent documents that install malware covertly on victims’ systems.
Unless you are running specialist monitoring tools, or very effective security software, you probably won’t see any symptoms of the attack.
The goals of these attacks are varied. In some cases they provide remote access to hackers. In others so-called cryptocurrency mining software is installed. These programs (ab)use your systems’ processing power in an attempt to generate cryptocurrencies such as Monero. The attackers get rich off your power bill.
While there are variations in how the attacks work, the typical path to compromise involves opening the document, which could be in Microsoft Word format, after which an exploit runs a PowerShell script. This, in turn, downloads and installs the malware.
Network security appliances
In this report we investigate how effectively some very popular network security products are at handling these and other threats.
As usual, we have also thrown in some particularly devious targeted attacks that appear to be completely legitimate applications but that provide us with remote access to unprotected targets. When we gain this access we try to hack the target in the same way a real attacker would. This gives the security products the best chance of detecting and potentially blocking the bad behaviour.
The good news is that all of these products were able to detect many (if not all) of the threats. Some were able to block most, although complete protection is not guaranteed. As always, a layered approach to protection is best. For advice on which endpoint software to choose see our Endpoint Protection test results on our website.
Latest report (PDF) now online.
Featured podcast:
All posts
What’s the difference between SE Labs and a cyber-criminal?
As we prepared this network security appliance report for publication we were also getting ready to present at BT’s internal security conference Snoopcon.
We had been asked to talk about security products and how they might not do what you assume they will.
Network Security Appliance report
Reports like this (PDF) provide an interesting insight into how security products actually work. Marketing messages will inevitably claim world-beating levels of effectiveness, while basic tests might well support these selling points. But when you actually hack target systems through security appliances you sometimes get a very different picture.
Some vendors will support the view that testing using a full attack chain (from a malicious URL pushing an exploit, which in turn delivers a payload that finally provides us with remote access to the system) is the right way to test. Others may point out that the threats we are using don’t exactly exist in the real world of criminality because we created them in the lab and are not using them to break into systems worldwide.
Available tools
We think that is a weak argument. If we can obtain access to certain popular, inexpensive tools online and create threats then these (or variants extremely close to them) are just as likely to exist in the ‘real world’ of the bad guys as in a legitimate, independent test lab. Not only that, but we don’t keep creating new threats until we break in, which is what the criminals (and penetration testers) do. We create a set and, without bias, expose all of the tested products to these threats.
But in some ways we have evolved from being anti-malware testers to being penetration testers, because we don’t just scan malware, execute scripts or visit URLs. Once we gain access to a target we perform the same tasks as a criminal would do: escalating privileges, stealing password hashes and installing keyloggers. The only difference between us and the bad guys is that we’re hacking our own systems and helping the security vendors plug the gaps.
Latest report (PDF) now online.
All posts
Hacked! Will your anti-malware protect you from targeted attacks?
Is anti-malware protection always strong? The news isn’t good. Discover your best options in our latest reports.
Latest reports now online.
Criminals routinely create ingenious scams and indiscriminate attacks designed to compromise the unlucky and, occasionally, foolish. But sometimes they focus on a specific target rather than casting a net wide in the hope of landing something interesting.
Targeted attacks can range from basic, like an email simply asking you to send some money to an account, through to extremely devious and technical. If you received an email from your accountant with an attached PDF or Excel spreadsheet would you open it?
Most would and all that then stands between them and a successful hack (because the email was a trick and contained a dodgy document that gives remote control to the attacker) is the security software running on their PC. How good is your anti-malware protection?
Anti-malware protection from different threats
In this test we’ve included indiscriminate, public attacks that come at victims from the web and via email, but we’ve also included some devious targeted attacks to see how well-protected potential victims would be.
We’ve not created any new types of threat and we’ve not discovered and used ‘zero day’ attacks. Instead we took tools that are freely distributed online and are well-known to penetration testers and criminals alike. We used these to generate threats that are realistic representations of what someone could quite easily put together to attack you or your business.
The results are extremely worrying. While a few products were excellent at detecting and protecting against these threats many more were less useful. We will continue this work and report any progress that these companies make in improving their products.
See if your anti-malware protection is up to scratch!
Our latest reports, for enterprise, small business and home users are now available for free from our website. Please download them and follow us on Twitter and/or Facebook to receive updates and future reports. Our blog is always here with extra help.
All posts
Network appliances vs. targeted attacks
There have been so many publicised data breaches in 2017 that we didn’t even have enough space in our latest report to provide a basic summary. In many cases a business network was breached. Business networks comprise endpoints (usually Windows PCs), servers, Point of Sale computers and a range of other devices.
Latest reports now online.
In this report (PDF) we explore the effectiveness of network appliances designed to detect and block attacks against endpoint systems. Who wins in the battle between network appliances vs. targeted attacks?
One approach to compromising a business is to hack an endpoint (PC) and then to use it as a platform from which to launch further attacks into the network. For example, rather than going straight for a company’s main servers why not trick a user into infecting his/ her computer with malware? We can then scan and infect the entire network, stealing information, causing damage and generally behaving in ways contrary to the business’ best interests.
There is some really good endpoint software available, as we see in our regular Endpoint Protection tests, but nothing is perfect and any extra layers of security are welcome. If one layer fails, others exist to mitigate the threat. In this report we explore the effectiveness of network appliances designed to detect and protect against attacks against endpoint systems.
Network appliances vs. targeted attacks
The systems we have tested here are popular appliances designed to sit between your endpoints and the internet router. They are designed to detect, and often protect against, threats coming in from the internet or passing through the local network. Their role is to stop threats before they reach the endpoints. If they fail to stop a threat, they might learn that an attack has happened and generate an alert, while subsequently blocking future, similar attacks.
There are no guarantees that technology will always protect you from attackers, but our results show that adding layers of security is an effective way to improve your prospects when facing general and more targeted attacks.
Featured podcast:
All posts
Anatomy of a Phishing Attack
We look at phishing attack tactics and impact. Who attacked a couple of internet pressure groups earlier this year? Let’s examine the evidence.
It is interesting to read about the public details of an unusually high-quality spear-phishing attack against a low value target. Particularly if you are engaged in constructing carefully-crafted tests of email security services.
Continue reading “Anatomy of a Phishing Attack”All posts
Next-generation firewalls: latest report
Using layers of security is a well-known concept designed to reduce the chances of an attacker succeeding in breaching a network. If one layer fails, others exist to mitigate the threat. Next-generation firewalls are a common protection layer. But how well do they work?
Latest reports now online.
