Sector: Enterprise
Holding Cyber Security to Account
It’s a phrase I hate: People are the weakest link in cyber security. Technology is supposed to serve humans, not the other way around. When we use computers in our personal and business lives, we have certain goals. Entertainment. Making money. Administering our energy bills, car insurance and any number of other important tasks. But our goals are probably not ‘security’.
Loading…
Reload document 
Open in new tab 
You are not the weakest link
We should be able to rely confidently on the security products that everyone tells us we need. The endpoint protection products in this report have undergone the most strenuous testing available, and they’ve come out well. They’ll provide you with strong protection while you use your computer to do something useful, fun or both.
How we test
We tested a variety of anti-malware (aka ‘anti-virus’; aka ‘endpoint security’) products from a range of well-known vendors in an effort to judge which were the most effective. Each product was exposed to the same threats, which were a mixture of targeted attacks using well-established techniques and public email and web-based threats that were found to be live on the internet at the time of the test. The results indicate how effectively the products were at detecting and/or protecting against those threats in real-time.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
Early Protection Systems
Testing protection against fully featured attacks
There are many opportunities to spot and stop attackers. Products can detect them when attackers send phishing emails to targets. Or later, when other emails contain links to malicious code. Some kick into action when malware enters the system. Others sit up and notice when the attackers exhibit bad behaviour on the network.
Regardless of which stages your security takes effect, you probably want it to detect and prevent before the breach runs to its conclusion in the press. Our Enterprise Advanced Security test is unique, in that we test products by running a full attack. We follow every step of a breach attempt to ensure that the test is as realistic as possible.
Loading…
Product factsheet:
Testing protection against fully featured attacks
Our Enterprise Advanced Security test is unique, in that we test products by running a full attack. We follow every step of a breach attempt to ensure that the test is as realistic as possible. This is important because different products can detect and prevent threats differently. In this report we look at how Coro – EDR handled full breach attempts. At which stages did it detect and protect? And did it allow business as usual, or mis-handle legitimate applications?
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
Endpoint Detection Compared
We compare endpoint security products directly using real, major threats
Welcome to the third edition of the Enterprise Advanced Security test, where we directly compare various endpoint security products. This report examines how these products tackle major threats faced by businesses of all sizes from the Global 100 down to medium enterprises, and likely small businesses too. While we provide an overall score, we also delve into the specific details that matter most to your security team, outlining the different levels of protection these products offer.
Endpoint Detection and Response (EDR) solutions go beyond traditional antivirus software, requiring more advanced testing methods. To truly evaluate EDR capabilities, testers need to act like real attackers, meticulously replicating each step of an attack.
It might be tempting to take shortcuts during testing, but to genuinely assess an EDR product’s effectiveness, it’s crucial to execute every stage of an attack. And each of these stages needs to be realistic; you can’t just guess what cybercriminals might do. That’s why SE Labs carefully tracks real-world cybercriminal behaviour and designs tests based on their tactics.
Thankfully, the MITRE organization has outlined these steps through its ATT&CK framework. While this framework doesn’t provide a precise guide for every attack scenario, it offers a valuable structure that testers, security vendors, and customers (like you!) can use to conduct tests and interpret results.
Loading…
How we test endpoint security products
We tested a variety of Endpoint Detection and Response products against a range of hacking attacks
designed to compromise systems and penetrate target networks in the same way criminals and other attackers breach systems and networks.
Full chains of attack were used, meaning that testers behaved as real attackers, probing targets using a variety of tools, techniques and vectors before attempting to gain lower-level and more powerful access. Finally, the testers/attackers attempted to complete their missions, which might include stealing information, damaging systems and connecting to other systems on the network.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
Put your focus on Business Email Compromise (BEC) scenarios
Don’t ignore Business Email Compromise test cases
Good security testing is realistic, using the kinds of threats customers see in real life. This is why we put a lot of focus on Business Email Compromise (BEC) scenarios, rather than just more conventional threat types (like generic phishing and malware).
Put focus on Business Email Compromise (BEC) scenarios
Many organisations focus on blocking spam and detecting malware, but BEC attacks present a different kind of threat. BEC targets the human element of email communication. Attackers craft convincing, fraudulent emails that appear to come from legitimate sources, tricking recipients into transferring money, sharing sensitive information or performing other actions that compromise the organisation. BEC cases are not about malware detection or basic spam filtering. Instead, they exploit trust and authority.
These attacks may bypass traditional security mechanisms because they often don’t contain malicious links or attachments. Instead, they rely on social engineering, making them incredibly dangerous and quite hard to spot by either people or technology.
Loading…
The cyber security industry refers to this sequence of steps as the ‘attack chain.’ The MITRE organization has documented these stages in its ATT&CK framework. While this framework doesn’t provide an exact blueprint for real-world attacks, it offers a structured guide that testers, security vendors, and customers (like you!) can use to conduct tests and interpret the results.
How we test
SE LABS Ⓡ tested three email security services, one that is commercial, the other open-source. We also tested a commercial email platform. Each service was exposed to the same threats, which were a mixture of targeted attacks using well-established techniques and public attacks that were found to be live on the internet at the time of the test.
The results indicate how effectively the services were at detecting and/or protecting against those threats in real-time and shortly after the attacks took place.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
Cyber Security Protection has Evolved
Top-tier anti-virus solutions are undeniably ‘next-generation’. This term was introduced nearly a decade ago by newcomers to the industry: a marketing device designed to compete with almost unassailable anti-malware brands.
“Stop using that tired old anti-virus and try the new, improved approach! No more mistakes. No updates. Full protection!” Problem solved. Except in 2024 I don’t think anyone would claim that malware is a thing of the past.
Cyber security protection has evolved and the leading vendors have embraced advanced technologies to stay ahead of increasingly sophisticated threats. Traditional anti-virus relies solely (or mostly) on signature-based detection. That’s not enough to defend against modern-day attacks like ransomware, file-less malware and zero-day exploits.
Loading…
Cyber security protection has evolved
Next-generation anti-virus is now the benchmark for the best security solutions. These systems go far beyond the old-fashioned model of looking for known malware signatures. Instead, they use a combination of machine learning, artificial intelligence and behavioural analysis to monitor how programs and processes behave in real-time.
How we test
We tested a variety of anti-malware (aka ‘anti-virus’; aka ‘endpoint security’) products from a range of well-known vendors in an effort to judge which were the most effective. Each product was exposed to the same threats, which were a mixture of targeted attacks using well-established techniques and public email and web-based threats that were found to be live on the internet at the time of the test. The results indicate how effectively the products were at detecting and/or protecting against those threats in real-time.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
All reports
09/2024 - 09/2024
Enterprise Advanced Security (EAS): Acronis Cyber Protect Cloud with Advanced Security + XDR Pack
Endpoint Detection and Response is more than anti-virus
Gain insights into cyber security testing through transparent threat intelligence
An Endpoint Detection and Response (EDR) product is much more than anti-virus which is why it requires more sophisticated testing. This involves testers mimicking real attackers and following every step of an attack.
How we test the effectiveness of Endpoint Detection and Response products
While shortcuts might seem tempting, fully executing each phase of an attack is crucial to truly evaluate the effectiveness of EDR products.
Moreover, each step must reflect real-world scenarios. You can’t just guess what cybercriminals might do and hope it’s accurate. That’s why SE Labs tracks the actual behaviour of cybercriminals and designs tests based on how attackers attempt to compromise their targets.
The cyber security industry refers to this sequence of steps as the ‘attack chain.’ The MITRE organization has documented these stages in its ATT&CK framework.
Loading…
How we tested Acronis Cyber Protect Cloud with Advanced Security + XDR Pack
We tested Acronis Cyber Protect Cloud with Advanced Security + XDR Pack against a range of hacking attacks designed to compromise systems and penetrate target networks in the same way as criminals and other attackers breach systems and networks. Full chains of attack were used, meaning that testers behaved as real attackers, probing targets using a variety of tools, techniques and vectors before attempting to gain lower-level and more powerful access. Finally, the testers/ attackers attempted to complete their missions, which might include stealing information, damaging systems and connecting to other systems on the network.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
Is AI able to protect your computer systems?
And are attackers using it to breach your network? Artificial Intelligence is ruling the stock market and may be on the verge of ruling the world if you believe the business influencers. If AI is as powerful as some say, surely it should be able to protect our computer systems from hackers?
The products in this test almost certainly rely on AI-related technologies to detect and protect against attacks. These technologies have been running in the background for about 20 years. We can argue that not only does anti-virus/ endpoint protection use AI, but it’s been doing so for many years, and certainly before Cylance claimed to be the first.
But I did something sneaky there. I slid in the word ‘-related’. Because when people talk about ChatGPT and other popular ‘AI’ tools, they are usually talking about something else. They are amazed by the utility of Machine Learning (ML) systems, which appear to be able to mimic human thought in a rather magical way.
Loading…
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
All reports
05/2024 - 05/2024
Enterprise Advanced Security (EAS): Acronis Cyber Protect Cloud with Advanced Security pack + EDR – DETECTION
Understand cyber security testing
An Endpoint Detection and Response (EDR) product is more than antivirus, which is why it requires advanced testing. This means testers must behave like real attackers, following every step of an attack. While it’s tempting to save time by taking shortcuts, a tester must go through an entire attack to truly understand cyber security testing and the capabilities of EDR security products.
Each step of the attack must be realistic too. You can’t just make up what you think bad guys are doing and hope you’re right. This is why SE Labs tracks cybercriminal behaviour and builds tests based on how bad guys try to compromise victims. The cybersecurity industry is familiar with the concept of the
‘attack chain’, which is the combination of those attack steps.
Fortunately, the MITRE organisation has documented each step with its ATT&CK framework. While this doesn’t give an exact blueprint for realistic attacks, it does present a general structure that testers, security vendors and customers (you!) can use to run tests and understand test results.
Loading…
Understand cyber security testing
You can see how ATT&CK lists out the details of each attack, and how we represent the way we tested, in 4. Threat Intelligence, starting on page 13. This brings two main advantages: you can have confidence that the way we test is realistic and relevant; and you’re probably already familiar with this way of illustrating cyber attacks
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
Lift your org’s security into the top 1%
Back to security basics, lift your org’s security into top 1%. It’s a shocking statistic, but most businesses don’t have a cyber security plan. Given the lack of general interest in cyber security, it’s no wonder that ransomware and less obvious threats are running riot through computer systems all over the world.
The larger the organisation, the more complex its security needs, but at the very core the same old cliched advice still applies to everyone:
- Apply security patches when available.
- Back up your important data.
- Use multi-factor authentication.
- Use endpoint protection.
If that sounds painfully basic and boring, I’m sorry. But it’s still the best advice out there. You don’t need AI, blockchain or other magical concepts to raise yourself to the top 1% of secure internet users.
If you want to know which endpoint protection to use, this report will help you or your business choose. There is, of course, more to cyber security than choosing an ‘anti-virus’ program though!
Loading…
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
All reports
06/2024 - 06/2024
Data Corruption Detection (DCD): NetApp ONTAP Autonomous Ransomware Protection with AI – DETECTION
Ransomware on the Network
Avoid ransomware on the network. Ransomware is the most visible, most easily understood cyber threat affecting businesses today. Paralysed computer systems mean stalled business and loss of earnings. On top of that, a ransom demand provides a clear, countable value to a threat. A demand for “one million dollars!” is easier to quantify than the possible leak of intellectual property to a competitor.
One reason why ransomware is so ‘popular’ is that the attackers don’t have to produce their own. They outsource the production of ransomware to others, who provide Ransomware as a Service (Raas). Attackers then usually trick targets into running it, or at least into providing a route for the attackers to run it for them. Artificial intelligence systems make the creation of such social engineering attacks easier, cheaper and more effective than ever before.
Ransomware attacks on the network
In this report we have assembled a wide distribution of known ransomware malware and added variations designed to evade detection. We’ve listed the ransomware families used in Threat Intelligence on page 9. Each of these ransomware attacks targeted data that was monitored by NetApp’s solution.
If it can detect the changes made by known version of each of these files, all well and good. But if it can also detect changes made by each of the ransomware’s variations then we can conclude that the detection available is more proactive than simply reacting to yesterday’s unlucky victims.
Loading…
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
