Sector: Enterprise
Is AI able to protect your computer systems?
And are attackers using it to breach your network? Artificial Intelligence is ruling the stock market and may be on the verge of ruling the world if you believe the business influencers. If AI is as powerful as some say, surely it should be able to protect our computer systems from hackers?
The products in this test almost certainly rely on AI-related technologies to detect and protect against attacks. These technologies have been running in the background for about 20 years. We can argue that not only does anti-virus/ endpoint protection use AI, but it’s been doing so for many years, and certainly before Cylance claimed to be the first.
But I did something sneaky there. I slid in the word ‘-related’. Because when people talk about ChatGPT and other popular ‘AI’ tools, they are usually talking about something else. They are amazed by the utility of Machine Learning (ML) systems, which appear to be able to mimic human thought in a rather magical way.
Loading…
Reload document 
Open in new tab 
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
All reports
05/2024 - 05/2024
Enterprise Advanced Security (EAS): Acronis Cyber Protect Cloud with Advanced Security pack + EDR – DETECTION
Understand cyber security testing
An Endpoint Detection and Response (EDR) product is more than antivirus, which is why it requires advanced testing. This means testers must behave like real attackers, following every step of an attack. While it’s tempting to save time by taking shortcuts, a tester must go through an entire attack to truly understand cyber security testing and the capabilities of EDR security products.
Each step of the attack must be realistic too. You can’t just make up what you think bad guys are doing and hope you’re right. This is why SE Labs tracks cybercriminal behaviour and builds tests based on how bad guys try to compromise victims. The cybersecurity industry is familiar with the concept of the
‘attack chain’, which is the combination of those attack steps.
Fortunately, the MITRE organisation has documented each step with its ATT&CK framework. While this doesn’t give an exact blueprint for realistic attacks, it does present a general structure that testers, security vendors and customers (you!) can use to run tests and understand test results.
Loading…
Understand cyber security testing
You can see how ATT&CK lists out the details of each attack, and how we represent the way we tested, in 4. Threat Intelligence, starting on page 13. This brings two main advantages: you can have confidence that the way we test is realistic and relevant; and you’re probably already familiar with this way of illustrating cyber attacks
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
Lift your org’s security into the top 1%
Back to security basics, lift your org’s security into top 1%. It’s a shocking statistic, but most businesses don’t have a cyber security plan. Given the lack of general interest in cyber security, it’s no wonder that ransomware and less obvious threats are running riot through computer systems all over the world.
The larger the organisation, the more complex its security needs, but at the very core the same old cliched advice still applies to everyone:
- Apply security patches when available.
- Back up your important data.
- Use multi-factor authentication.
- Use endpoint protection.
If that sounds painfully basic and boring, I’m sorry. But it’s still the best advice out there. You don’t need AI, blockchain or other magical concepts to raise yourself to the top 1% of secure internet users.
If you want to know which endpoint protection to use, this report will help you or your business choose. There is, of course, more to cyber security than choosing an ‘anti-virus’ program though!
Loading…
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
All reports
06/2024 - 06/2024
Data Corruption Detection (DCD): NetApp ONTAP Autonomous Ransomware Protection with AI – DETECTION
Ransomware on the Network
Avoid ransomware on the network. Ransomware is the most visible, most easily understood cyber threat affecting businesses today. Paralysed computer systems mean stalled business and loss of earnings. On top of that, a ransom demand provides a clear, countable value to a threat. A demand for “one million dollars!” is easier to quantify than the possible leak of intellectual property to a competitor.
One reason why ransomware is so ‘popular’ is that the attackers don’t have to produce their own. They outsource the production of ransomware to others, who provide Ransomware as a Service (Raas). Attackers then usually trick targets into running it, or at least into providing a route for the attackers to run it for them. Artificial intelligence systems make the creation of such social engineering attacks easier, cheaper and more effective than ever before.
Ransomware attacks on the network
In this report we have assembled a wide distribution of known ransomware malware and added variations designed to evade detection. We’ve listed the ransomware families used in Threat Intelligence on page 9. Each of these ransomware attacks targeted data that was monitored by NetApp’s solution.
If it can detect the changes made by known version of each of these files, all well and good. But if it can also detect changes made by each of the ransomware’s variations then we can conclude that the detection available is more proactive than simply reacting to yesterday’s unlucky victims.
Loading…
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
All reports
05/2024 - 05/2024
Email Security Services (ESS): Microsoft Defender for Office 365 2024 Q2
Email Security Is Essential
Email is one of the most common ways that threats will hit an organisation. It’s the first stage in a series of unpleasant, expensive events that leads to data theft, data destruction and business cessation. Email is one of the very few standard ways that hackers start their attacks, this is why email security is so essential.
Classic examples of email threats include phishing emails, designed to steal important information that aid deeper attacks. Emails can contain links to dangerous websites that can trick users into handing over critical information or may even directly attack the user’s computer. Attached documents may contain nasty surprises, such as backdoors that give attackers access to the business’ network. Access means theft and destruction (e.g. ransomware).
Loading…
If the email security service you use can stop most of that, it massively reduces the risk from hacking. Not using one is, frankly, irresponsible.
You cannot just plug in email security or rely on the security features provided by your email platform, though. Configuration is king. Given that most businesses in the UK and USA don’t have a cyber security plan, it’s likely that many Office 365 users have not changed their email security settings from the default. In this report we used Microsoft’s best practice configuration, rather than the default. Even then, after the test, Microsoft recommended changes.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
Essential Endpoint Security Enterprise
Realistic attacks for useful results
Essential Endpoint Security for Enterprise. Keeping your organisation safe from online threats requires strong endpoint protection. It’s not just important – it’s crucial. So, it’s a good idea to regularly check how well it’s working. Essential endpoint security means checking if the security tools on devices like desktops, laptops, and mobiles do their job.
Why? Because these devices are often the target of online attacks. If they’re not protected, they can become a way for cybercriminals to get to your sensitive information.
Our reports help you choose the best home anti-malware product that can protect you from ransomware and other types of attacks.
Loading…
How we test
SE Labs brings a wealth of experience to the table when it comes to testing endpoint protection. We firmly believe in the necessity of conducting these tests regularly to ensure that security vendors are consistently updating and enhancing their effectiveness. Our testing approach involves recreating real-world cyberattack situations, allowing us to assess the performance of endpoint security solutions in terms of detection, prevention, and mitigation.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
Ransomware vs. Endpoint Security
Ransomware vs. Endpoint Security – Results from the largest public ransomware test
In this report, we analyse ransomware vs. endpoint security. Ransomware is the most visible, most easily understood cyber threat affecting businesses today. Paralysed computer systems mean stalled business and loss of earnings. On top of that, a ransom demand provides a clear, countable value to a threat. A demand for “one million dollars!” is easier to quantify than the possible leak of intellectual property to a competitor.
Loading…
Product factsheet:
In this report, we have taken two main approaches to assessing how well products can detect and protect against ransomware.
Ransomware Deep Attacks
For the first part of this test, we analysed the common tactics of ransomware gangs and created two custom gangs that use a wider variety of methods. In all cases, we run the attack from the very start, including attempting to access targets with stolen credentials or other means. We then move through the system and sometimes the network, before deploying the ransomware as the final payload.
Ransomware Direct Attacks
The second part of the test takes a wide distribution of known malware and adds variations designed to
evade detection. We’ve listed the ransomware families used in Hackers vs. Targets on page 9. We sent each of these ransomware payloads directly to target systems using realistic techniques, such as through
email social engineering attacks. This is a full but short attack chain. In this part of the test, we ensure any protection features are enabled in the product.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
Critical Endpoint Protection
Realistic attacks for useful results
Endpoint protection is a critical component of any organisation’s cyber security strategy
And if it’s critical then you should test it. And have others run assessments too. Testing endpoint protection involves evaluating the effectiveness of the security solutions. These solutions are deployed on endpoint devices such as desktops, laptops and mobile devices. Endpoint protection testing is necessary because endpoints are often the target of cyberattacks. Endpoints can be used as a gateway to gain access to sensitive data.
Our reports help you choose the best home anti-malware product that can protect you from ransomware and other types of attack.
Loading…
SE Labs has extensive experience in endpoint protection testing. We have found that it is essential to conduct these tests regularly to ensure that the security vendors are keeping them up-to-date and effective. Our testing methodology involves replicating real-world cyberattack scenarios. And then evaluating how well the endpoint security solutions perform in detecting, preventing, and mitigating those attacks.
One of the key aspects of endpoint protection testing is the use of realistic attack scenarios. Our team of experts analyses the latest threat intelligence and creates attack scenarios that closely mimic the tactics, techniques, and procedures used by real-world cybercriminals. By doing so, we can determine how well the endpoint security solutions perform in detecting and preventing these attacks.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
Content Disarm and Reconstruction (CDR)
Content Disarm and Reconstruction (CDR)
CDR tested for removal of known and unknown threats
SE Labs tested OPSWAT Deep CDR (Content Disarm and Reconstruction) against targeted attacks using file-based threats. These attacks are designed to compromise systems and penetrate target networks by hiding threats inside files that appear to be innocent.
Testers hid threats inside a variety of common file formats, such as office documents, web pages and archive files.
Content Disarm and Reconstruction (CDR) vs. hidden threats
These files were assessed by the CDR system, which attempted to remove known and unknown threats. The results show the extent to which the threat prevention system achieved that goal accurately.
Read more reports here.
Loading…
Product factsheet:
CDR solutions work differently than traditional solutions
CDR security solutions take a different approach than many others. Instead of detecting threats, they pull files apart and put them back together again. The idea is that anything bad gets dropped by the wayside, and only good things can pass through.
This approach is particularly appropriate when considering the risk of man-in-the-middle attacks. You might send a useful file to a colleague, but an attacker intercepts it and adds a little extra something, like a remote access tool. When you open it, you see what you would expect, while the attacker gains access to your system.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
All reports
09/2023 - 09/2023
Enterprise Advanced Security (EDR): Cisco Secure Endpoint – PROTECTION
Cisco Secure Endpoint
Testing protection against fully featured attacks
SE Labs tested Cisco Secure Endpoint – Protection against targeted attacks based on the Turla threat. These attacks are designed to compromise systems and penetrate target networks in the same way as the advanced persistent hacking group known as Turla operates to breach systems and networks.
There are many opportunities to spot and stop attackers. Products can detect them when attackers send phishing emails to targets. Or later, when other emails contain links to malicious code. Some kick into action when malware enters the system. Others sit up and notice when the attackers exhibit bad behaviour on the network.
Ultimately you want your chosen security product to prevent a breach one way or another, but it’s more ideal to stop a threat early, rather than watch as it wreaks havoc before stopping it and trying to clean up.
Loading…
Product factsheets:
Results
Cisco Secure Endpoint scored a 100% Protection Accuracy Rating for blocking every threat at the initial delivery stage. The product did not generate any false positives, meaning that it didn’t wrongly detect or hamper harmless, legitimate software.
It also prevented all of the malicious activities from running, incurring no penalties for allowing the full or partial execution of targeted attacks.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
