Sector: Enterprise
All reports
04/2025 - 05/2025
Advanced Security Test Report: Palo Alto Networks Cortex XDR – EDR (Protection)
Ransomware vs. Endpoint Security
Palo Alto Networks Cortex XDR test results by SE Labs.
SE LABS tested Palo Alto Networks Cortex XDR against a range of ransomware attacks designed to extort victims. These attacks were realistic, using the same tactics and techniques as those used against victims in recent months.
We attacked target systems, protected by Cortex XDR, acting in the same way as we observe ransomware groups to behave.
Attacks were initiated from the start of the attack chain, using phishing email links and attachments, as just two examples. Each attack was run from the very start to its obvious conclusion, which means attempting to steal, encrypt and destroy sensitive data on the target systems.
Loading…
Reload document 
Open in new tab 
Ransomware is the most visible, most easily understood cyber threat affecting businesses today. Paralysed computer systems mean stalled business and loss of earnings. On top of that, a ransom demand provides a clear, countable value to a threat. A demand for “one million dollars!” is easier to quantify than the possible leak of intellectual property to a competitor.
One reason why ransomware is so ‘popular’ is that the attackers don’t have to produce their own. They outsource the production of ransomware to others, who provide Ransomware as a Service (RAAS). Attackers then usually trick targets into running it, or at least into providing a route for the attackers to run it for them. Artificial intelligence systems make the creation of such social engineering attacks easier, cheaper and more effective than ever before.
Given the global interest and terror around ransomware, we have created a comprehensive test that shows how effective security products are when faced with the whole range of threats posed by ransomware itself and the criminal groups operating in the shadows.
In this report we have taken two main approaches to assessing how well products can detect and protect against ransomware.
Ransomware Deep Attacks
For the first part of this test, we analysed the common tactics of ransomware gangs and created two custom gangs that use a wider variety of methods. In all cases we run the attack from the very start, including attempting to access targets with stolen credentials or other means. We then move through the system and sometimes the network, before deploying the ransomware as the final payload.
In the first two attacks for each group, we gain access and deploy ransomware onto the target immediately. In the third, fourth and fifth attacks we move through the network and deploy ransomware on a target deeper into the network.
The ransomware payloads we used in this part of the report were known files from all of the families listed in Attack Details on page 8.
This test shows a product’s ability to track the movement of the attacker through the entire attack chain. We disable the product’s protection features and rely on its detection mode for this part of the test. The results demonstrate how incident response teams can use the product to gain visibility on ransomware attacks.
Ransomware Direct Attacks
The second part of the test takes a wide distribution of known malware and adds variations designed to evade detection. We’ve listed the ransomware families used in Attack Details on page 8. We sent each of these ransomware payloads directly to target systems. We used realistic techniques, such as through email social engineering attacks. This is a full but short attack chain.
In this part of the test, we ensure any protection features are enabled in the product.
If products like Palo Alto Networks Cortex XDR can detect and protect against the known version of each of these files, all well and good. But if they also detect and block each ransomware’s two variations, we can conclude that the protection available is more proactive than simply reacting to yesterday’s unlucky victims.
All reports
04/2025 - 06/2025
Security Evaluation Test Report: Enterprise Endpoint Security (Protection)
The Cost of Enterprise Endpoint Protection Failure
It will always be more than the cost of good protection. Whether you provide security for a global enterprise or run a small business with just a few employees, a single compromised endpoint brings serious consequences. In many cases, attackers don’t breach the most valuable system, but the most vulnerable. Once breached, attackers can move on to steal data, disrupt operations or deploy ransomware that stops business in its tracks.
Loading…
It will always be more than the cost of good protection
For large organisations, the impact might include fines, reputational damage and widespread operational
downtime.
For smaller companies, the effect can be far worse. A single ransomware incident or business
email compromise could lead to a level of financial loss that the business cannot absorb. In some cases,
it means closure.
The Cost of Enterprise Endpoint Protection Failure
Why do we go to all this trouble? Because businesses need answers grounded in reality, not synthetic benchmarks or scripted demos. We copy the bad guys to discover the truth.
These include common malware found in the wild and more advanced attacks modelled on real adversaries. Some threats were captured directly from the internet and tested immediately. Others were designed to reflect how a capable attacker behaves, using techniques such as spear phishing and running post-exploitation tools within a network.
Which solutions to trust?
Effective endpoint protection must do more than respond to known threats. It must adapt quickly, stop attacks early and resist attempts to bypass defences. While no product is perfect, some provide a much higher level of protection than others. This report makes those differences clear.
How we test
We tested a variety of anti-malware (aka ‘anti-virus’; aka ‘endpoint security’) products from a range of well-known vendors in an effort to judge which were the most effective. Each product was exposed to the same threats, which were a mixture of targeted attacks using well-established techniques and public email and web-based threats that were found to be live on the internet at the time of the test. The results indicate how effectively the products were at detecting and/or protecting against those threats in real-time.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
All reports
03/2025 - 04/2025
Advanced Security Test Report: Cisco Secure Email Threat Defense – Email (Protection)
Ignore Business Email Compromise test cases at your peril
Cisco Secure Email Threat Defense Protection test results.
SE LABS tested Cisco Secure Email Threat Defense, against a mixture of targeted attacks using well-established techniques and public attacks that were found to be live on the internet at the time of the test.
The results indicate how effectively the service was at detecting and/or protecting against those threats in real time and shortly after the attacks took place.
Loading…
Good security testing is realistic, using the kinds of threats customers see in real life. This is why we put a lot of focus on Business Email Compromise (BEC) scenarios, rather than just more conventional threat types (like generic phishing and malware).
Many organisations focus on blocking spam and detecting malware, but BEC attacks present a different kind of threat. BEC targets the human element of email communication. Attackers craft convincing, fraudulent emails that appear to come from legitimate sources, tricking recipients into transferring money, sharing sensitive information or performing other actions that compromise the organisation.
BEC attacks exploit trust
BEC cases are not about malware detection or basic spam filtering. Instead, they exploit trust and authority. These attacks may bypass traditional security mechanisms because they often don’t contain malicious links or attachments. Instead, they rely on social engineering, making them incredibly dangerous and quite hard to spot by either people or technology.
Testing email security without BEC scenarios is to ignore a highly effective and popular method that attackers use every day to infiltrate businesses. It’s essential to ensure that email security solutions are able to recognise these nuanced threats and react accordingly.
Furthermore, adding security to a standard email platform shouldn’t be an afterthought. Many businesses assume that the platforms they use, such as Microsoft 365 or Google Workspace, have robust, built-in defences. While these platforms offer a solid baseline, they are not infallible. Attackers continuously evolve their tactics, exploiting gaps in standard security settings.
Cisco Secure Email Threat Defense
Comprehensive email security requires layered defences that integrate seamlessly with these platforms, providing advanced detection capabilities, including AI-driven anomaly detection, BEC filtering, and more.
By enhancing the built-in security of these platforms with products such as Cisco‘s, organisations can mitigate risks more effectively. Security should be adaptive and proactive, not reactive, ensuring that your organisation stays protected even as threats evolve. Including BEC scenarios in testing is an essential part of validating these systems’ robustness.
Network testing Standards, analysis and transparency
SE LABS tested the performance of the Cisco Secure Firewall 4225. We assessed its ability to operate under a variety of network loads, including a range of well-established but synthetic sets of traffic. We also used a more realistic mix of protocols.
This test is based on available standards of testing, including the methodology provided by the Internet Engineering Task Force. The results indicate how effectively the product was at handling network traffic in different circumstances, using the configuration specified.
Loading…
This test was conducted using recommendations made by the Internet Engineering Task Force for testing the performance of network security devices. Devices such as next-generation firewalls, intrusion detection and protection systems and unified threat management devices. At a minimum it should show, in a transparent and repeatable way, how the device under test handles network traffic of different types and in different scenarios.
On its own, the raw data is useful for comparing products with a view to choosing which is most suitable for your organisation.
We also ran an extended set of tests to see how a device would behave in a more realistic, production environment. This involved using a mixture of network traffic protocols and testing individual types of application traffic with the Cisco Secure Firewall 4225.
Cisco Secure Firewall 4225 performance test results
At SE Labs we don’t just publish raw figures, though. We use our knowledge and expertise to analyse that information to help add useful colour to the results.
The goal is to give a real-world opinion as to which figures are most important. We want to highlight where optimum performances are achieved and to explain why some details are more significant than others.
For example, a device might achieve an apparently strong performance when handling Voice over IP, but in real-life the human ear might struggle with sub-par connection quality. Conversely, what may seem like poor performance on paper might not be noticeable to users in a real deployment.
Testing standards and configurations
We have followed the available testing standards. This means you can verify our figures with reports generated by other test labs. It also gives you confidence that the testing was conducted correctly, while also being completely transparent about the configuration used. This configuration might not be the one you experience out of the box. Or might not be suitable for your own deployment. We’ve included configuration details so you can make a fully informed decision when comparing products and reports. See our full network performance testing methodology for more details.
All reports
05/2025 - 05/2025
Advanced Security Test Report: Cisco Universal ZTNA – IAM (Protection)
Managing identity and access in the real world
Cisco Universal ZTNA test results by SE Labs.
SE LABS tested Cisco Universal Zero Trust Network Access (ZTNA) against a range of attacks designed to bypass traditional security controls.
These attacks were realistic, using the same tactics and techniques as those used against victims in recent months.
Testers attacked target systems protected by Cisco Universal ZTNA. These same testers acted in the same way as we observe advanced attack groups to behave.
Attacks initiated from the start of the attack chain, using stolen credentials, multi-factor flooding techniques and hijacked sessions.
Each attack was run from the very start to its obvious conclusion, which means attempting to steal, encrypt and destroy sensitive data on the target systems.
This report is the first independent assessment of a service that aims to make it more secure to connect to cloud services.
If you’re interested in IAM, SASE and ZTNA then this is the report for you. We’ll explain what all those terms mean here.
The general term for ensuring that only the right people (or computers) can gain access is called Identity and Access Management (IAM). This is an IAM security report, but you could just as well call it an Identity Threat Detection and Response test.
SASE: Beyond the VPN
Many organisations use services in the cloud, needing high-performance, well-controlled authentication. Data needs to be accessible, at high speeds, but using strong security.
And businesses need to manage this security simply.
Secure access used to be handled primarily by Virtual Private Networks (VPNs). Demand for faster, more flexible approaches means that we’re more likely to connect to a well-distributed cloud service.
This is what SASE means – Secure Access Service Edge. It offers Secure Access, providing this Service at a location close to the user, at the ‘Edge’ of the internet. SASE includes a number of services. The one we’re interested in here is authentication.
Zero Trust
A Zero Trust Network Access (ZTNA) approach to authentication means, simply, that a system should never trust another system. It requires verification every time access is needed. This approach often includes the use of multi-factor authentication (MFA). However, it can become much more advanced and look at context. Such context coudl be “why is Simon logging in from London and Indonesia at the same time?”
Does Cisco Universal ZTNA Work?
Cisco’s Universal Zero Trust Network Access (UZTNA) is a solution combining multiple products to provide zero trust authentication. In this test we tested like hackers, attempting to break in using different techniques from different part of the world. We are proud to present the results in this report.
All reports
03/2025 - 04/2025
Advanced Security Test Report: Coro Email and Cloud Security – Email (Protection)
Email security vs. business-focussed attackers
SE LABS tested Coro Email and Cloud Security against a mixture of targeted attacks using well-established techniques and public attacks that were found to be live on the internet at the time of the test.
The results indicate how effectively the service was at detecting and/or protecting against those threats in real time and shortly after the attacks took place.
Loading…
Good security testing is realistic, using the kinds of threats customers see in real life. This is why we put a lot of focus on Business Email Compromise (BEC) scenarios, rather than just more conventional threat types (like generic phishing and malware).
Many organisations focus on blocking spam and detecting malware, but BEC attacks present a different kind of threat. BEC targets the human element of email communication. Attackers craft convincing, fraudulent emails that appear to come from legitimate sources, tricking recipients into transferring money, sharing sensitive information or performing other actions that compromise the organisation.
BEC cases are not about malware detection or basic spam filtering. Instead, they exploit trust and authority. These attacks may bypass traditional security mechanisms because they often don’t contain malicious links or attachments. Instead, they rely on social engineering, making them incredibly dangerous and quite hard to spot by either people or technology.
Coro Email and Cloud Security Protection test results
Testing email security, like that from Coro, without BEC scenarios is to ignore a highly effective and popular method that attackers use every day to infiltrate businesses. It’s essential to ensure that email security solutions are able to recognise these nuanced threats and react accordingly.
Furthermore, adding security to a standard email platform shouldn’t be an afterthought. Many businesses assume that the platforms they use, such as Microsoft 365 or Google Workspace, have robust, built-in defences. While these platforms offer a solid baseline, they are not infallible. Attackers continuously evolve their tactics, exploiting gaps in standard security settings.
Comprehensive email security requires layered defences that integrate seamlessly with these platforms, providing advanced detection capabilities, including AI-driven anomaly detection, BEC filtering, and more.
By enhancing the built-in security of these platforms, organisations can mitigate risks more effectively. Security should be adaptive and proactive, not reactive, ensuring that your organisation stays protected even as threats evolve. Including BEC scenarios in testing is an essential part of validating these systems’ robustness. See our full email testing methodology for more details.
Early warnings for targeted attacks
Fortinet FortiEDR Protection test results by SE LABS (Threat Series: 11).
These attacks are designed to compromise systems and penetrate target networks in the same way as the advanced persistent hacking groups known as Gamaredon Group, Ember Bear, Evasive Panda, and DPRK operate to breach systems and networks.
SE Labs used full chains of attack, meaning that testers behaved as real attackers, probing targets using a variety of tools, techniques and vectors before attempting to gain lower-level and more powerful access. Finally, the testers/attackers attempted to complete their missions, which might include stealing information, damaging systems and connecting to other systems on the network.
Loading…
There are many opportunities to spot and stop attackers. Products can detect them when attackers send phishing emails to targets. Or later, when other emails contain links to malicious code. Some kick into action when malware enters the system. Others sit up and notice when the attackers exhibit bad behaviour on the network.
Regardless of which stages your security takes effect, you probably want it to detect and prevent before the breach runs to its conclusion in the press.
Our Advanced Security test is unique, in that we test products by running a full attack. We follow every step of a breach attempt to ensure that the test is as realistic as possible.
This is important because different products can detect and prevent threats differently.
Ultimately you want your chosen security product to prevent a breach one way or another, but it’s more ideal to stop a threat early, rather than watch as it wreaks havoc before stopping it and trying to clean up.
Fortinet FortiEDR Protection test results
Some products are designed solely to watch and inform, while others can also remove threats either as soon as they appear or after they start causing damage.
For the ‘watchers’ we run the Advanced Security test in Detection mode. For ‘stoppers’ like Fortinet FortiEDR we can demonstrate effectiveness by testing in Protection Mode.
In this report we look at how Fortinet FortiEDR handled full breach attempts. At which stages did it detect and protect? And did it allow business as usual, or mis-handle legitimate applications?
Understanding the capabilities of different security products is always better achieved before you need to use them in a live scenario. SE Labs’ Advanced Security test reports help you assess which are the best for your own organisation.
All reports
04/2025 - 05/2025
Advanced Security Test Report: Acronis Cyber Protect Cloud with Advanced Security + XDR Pack – EDR (Detection)
Endpoint Detection and Response is more than anti-virus
Acronis Cyber Protect Cloud with Advanced Security + XDR Pack Detection test results by SE LABS (Threat Series: 11).
SE LABS tested Acronis Cyber Protect Cloud with Advanced Security + XDR Pack against a range of hacking attacks designed to compromise systems and penetrate target networks in the same way as criminals and other attackers breach systems and networks.
Full chains of attack were used, meaning that testers behaved as real attackers, probing targets using a variety of tools, techniques and vectors before attempting to gain lower-level and more powerful access. Finally, the testers/attackers attempted to complete their missions, which might include stealing information, damaging systems and connecting to other systems on the network.
Loading…
An Endpoint Detection and Response (EDR) product goes beyond traditional antivirus software, which is why it requires more sophisticated testing. This involves testers mimicking real attackers and following every step of an attack.
While shortcuts might seem tempting, fully executing each phase of an attack is crucial to truly evaluate the effectiveness of EDR products.
Moreover, each step must reflect real-world scenarios; you can’t just guess what cyber criminals might do and hope it’s accurate. That’s why SE Labs tracks the actual behaviour of cyber criminals and designs tests based on how attackers attempt to compromise their targets.
The cyber security industry refers to this sequence of steps as the ‘attack chain.’ The MITRE organization has documented these stages in its ATT&CK framework.
While this framework doesn’t provide an exact blueprint for real-world attacks, it offers a structured guide that testers, security vendors, and customers (like you!) can use to conduct tests and interpret the results.
Acronis Cyber Protect Cloud with Advanced Security + XDR Pack Detection test results
SE Labs’ Advanced Security tests are based on real attacker behaviour, and we present our findings using a MITRE ATT&CK-style format.
You can see how the ATT&CK framework outlines each step of an attack and how we apply it to our testing in section 4. Threat Intelligence, starting on page 12. This approach offers two key benefits: confidence that our tests are both realistic and relevant, and familiarity with the way cyber attacks are illustrated.
Check out this in-depth report on Acronis Cyber Protect Cloud with Advanced Security + XDR Pack.
All reports
03/2025 - 04/2025
Advanced Security Test Report: Symantec Endpoint Security Complete – EDR (Protection)
Ransomware vs. Endpoint Security
This is the most comprehensive test that shows how effective security products are when faced with the whole range of threats posed by ransomware itself and the criminal groups operating in the shadows.
In this report we have taken two main approaches to assessing how well products can detect and protect against ransomware.
Loading…
Ransomware is the most visible, most easily understood cyber threat affecting businesses today. Paralysed computer systems mean stalled business and loss of earnings. On top of that, a ransom demand provides a clear, countable value to a threat. A demand for “one million dollars!” is easier to quantify than the possible leak of intellectual property to a competitor.
One reason why ransomware is so ‘popular’ is that the attackers don’t have to produce their own. They outsource the production of ransomware to others, who provide Ransomware as a Service (RAAS).
Attackers then usually trick targets into running it, or at least into providing a route for the attackers to run it for them. Artificial intelligence systems make the creation of such social engineering attacks easier, cheaper and more effective than ever before.
Given the global interest and terror around ransomware, we have created a comprehensive test that shows how effective security products are when faced with the whole range of threats posed by ransomware itself and the criminal groups operating in the shadows.
In this report we have taken two main approaches to assessing how well products can detect and protect against ransomware.
Ransomware Deep Attacks
For the first part of this test, we analysed the common tactics of ransomware gangs and created two custom gangs that use a wider variety of methods. In all cases we run the attack from the very start, including attempting to access targets with stolen credentials or other means. We then move through the system and sometimes the network, before deploying the ransomware as the final payload.
In the first two attacks for each group, we gain access and deploy ransomware onto the target immediately. In the third, fourth and fifth attacks we move through the network and deploy ransomware on a target deeper into the network.
The ransomware payloads used in this part of the report were known files from all of the families listed in Attack Details on page 8. This test shows a product’s ability to track the movement of the attacker through the entire attack chain. We disable the product’s protection features and rely on its detection mode for this part of the test. The results demonstrate how incident response teams can use the product to gain visibility on ransomware attacks.
Ransomware Direct Attacks
The second part of the test takes a wide distribution of known malware and adds variations designed to evade detection. We’ve listed the ransomware families used in Attack Details on page 8. We sent each of these ransomware payloads directly to target systems using realistic techniques, such as through email social engineering attacks. This is a full but short attack chain. In this part of the test, we ensure any protection features are enabled in the product.
If products can detect and protect against the known version of each of these files, all well and good. But if they also detect and block each ransomware’s two variations then we can conclude that the protection available is more proactive than simply reacting to yesterday’s unlucky victims.
Ransomware vs. Endpoint Security
This is the most comprehensive test that shows how effective security products are when faced with the whole range of threats posed by ransomware itself and the criminal groups operating in the shadows.
In this report we have taken two main approaches to assessing how well products can detect and protect against ransomware.
Loading…
Ransomware is the most visible, most easily understood cyber threat affecting businesses today. Paralysed computer systems mean stalled business and loss of earnings. On top of that, a ransom demand provides a clear, countable value to a threat. A demand for “one million dollars!” is easier to quantify than the possible leak of intellectual property to a competitor.
One reason why ransomware is so ‘popular’ is that the attackers don’t have to produce their own. They outsource the production of ransomware to others, who provide Ransomware as a Service (RAAS).
Attackers then usually trick targets into running it, or at least into providing a route for the attackers to run it for them. Artificial intelligence systems make the creation of such social engineering attacks easier, cheaper and more effective than ever before.
Given the global interest and terror around ransomware, we have created a comprehensive test that shows how effective security products are when faced with the whole range of threats posed by ransomware itself and the criminal groups operating in the shadows.
In this report we have taken two main approaches to assessing how well products can detect and protect against ransomware.
Ransomware Deep Attacks
For the first part of this test, we analysed the common tactics of ransomware gangs and created two custom gangs that use a wider variety of methods. In all cases we run the attack from the very start, including attempting to access targets with stolen credentials or other means. We then move through the system and sometimes the network, before deploying the ransomware as the final payload.
In the first two attacks for each group, we gain access and deploy ransomware onto the target immediately. In the third, fourth and fifth attacks we move through the network and deploy ransomware on a target deeper into the network.
The ransomware payloads used in this part of the report were known files from all of the families listed in Attack Details on page 8. This test shows a product’s ability to track the movement of the attacker through the entire attack chain. We disable the product’s protection features and rely on its detection mode for this part of the test. The results demonstrate how incident response teams can use the product to gain visibility on ransomware attacks.
Ransomware Direct Attacks
The second part of the test takes a wide distribution of known malware and adds variations designed to evade detection. We’ve listed the ransomware families used in Attack Details on page 8. We sent each of these ransomware payloads directly to target systems using realistic techniques, such as through email social engineering attacks. This is a full but short attack chain. In this part of the test, we ensure any protection features are enabled in the product.
If products can detect and protect against the known version of each of these files, all well and good. But if they also detect and block each ransomware’s two variations then we can conclude that the protection available is more proactive than simply reacting to yesterday’s unlucky victims.
