Do you really understand AV test results?

[Andy Ful]

The things we are discussing here - like Ai and Machine Learning, were the promises that were made 20 years ago. Security fundamentals have always worked. Stick to the basics. The basics will take you further into the future than anything else. 1 + 1 = 2. If you block it from executing in the first place, then you do not have deal with any unpleasant consequences afterwards. Ai, Machine Learning, behavioral analysis, etc - they require you to execute something. If they don't catch it, you're beat ! You're much better off blocking by default. Default deny will save you much pain. So plainly obvious and common sense, you have to wonder why so very few people actually adhere to it.

It is a very well accepted fact in the healthcare. The Government can save a lot of people and money when investing in successful disease prevention, instead of fighting the disease when people will get sick.

 

[509322]

It is a very well accepted fact in the healthcare. The Government can save a lot of people and money when investing in successful disease prevention, instead of fighting the disease when people will get sick.

Even a lot of security soft geeks will not use software restriction policy to lock down their systems; they want default allow internet security suite or antivirus. They won't change until disaster strikes, and even then they still might not change. Just like some people will not stop eating a pound of bacon every day until they have a massive heart attack. There is no accounting for stupid human behavior. Doctors tell us we're all stupid, all of the time.

 

[Azure]

Thanks. I think I got it now:
the first unfortunate user, the one who got infected, actually saves everyone else. Because there is POST-infection Ai detection, and a cloud data base to push it in real-time to everyone using that AV. Please correct any mistakes here.

Which major AVs have post infection Ai detection? Or should I ask which ones do not have?

Microsoft has it
Detonating a bad rabbit: Windows Defender Antivirus and layered machine learning defenses

"Within a few seconds, the file was processed, and sample-analysis-based ML models returned their conclusions. In this case, a multi-class deep neural network (DNN) machine learning classifier correctly classified the Tibbar sample as malware, but with only an 81.6% probability score. In order to avoid false positives, cloud protection service was configured to require at least 90% probability to block the malware (these thresholds are continually evaluated and fine-tuned to find the right balance between blocking malware while avoiding the blocking of legitimate programs). In this case, the ransomware was allowed to run."
(Here we have the first user)

"In the meantime, while patient zero and eight other unfortunate victims (in Ukraine, Russia, Israel, and Bulgaria) contemplated whether to pay the ransom, the sample was being detonated and details of the system changes made by the ransomware recorded.
As soon as the detonation results were available, a multi-class deep neural network (DNN) classifier that used both static and dynamic features evaluated the results and classified the sample as malware with 90.7% confidence, high enough for the cloud to start blocking."
(And here Defender is able to protect subsequent users)

 

[Andy Ful]

Microsoft has it
Detonating a bad rabbit: Windows Defender Antivirus and layered machine learning defenses

"Within a few seconds, the file was processed, and sample-analysis-based ML models returned their conclusions. In this case, a multi-class deep neural network (DNN) machine learning classifier correctly classified the Tibbar sample as malware, but with only an 81.6% probability score. In order to avoid false positives, cloud protection service was configured to require at least 90% probability to block the malware (these thresholds are continually evaluated and fine-tuned to find the right balance between blocking malware while avoiding the blocking of legitimate programs). In this case, the ransomware was allowed to run."
(Here we have the first user)

"In the meantime, while patient zero and eight other unfortunate victims (in Ukraine, Russia, Israel, and Bulgaria) contemplated whether to pay the ransom, the sample was being detonated and details of the system changes made by the ransomware recorded.
As soon as the detonation results were available, a multi-class deep neural network (DNN) classifier that used both static and dynamic features evaluated the results and classified the sample as malware with 90.7% confidence, high enough for the cloud to start blocking."
(And here Defender is able to protect subsequent users)

Thanks for the link. So indeed, Microsoft AI can create a post-infection signature in minutes. What a strange time correlation with my thread-opening post - the article was published today. I wonder how this layered protection will work in the real world.
From this article, it follows that 'cloud protection level' Defender setting, can have an influence on the creation time of signatures in the cloud. In the higher setting, the malware could be stopped in a few seconds (not executed) - of course, this setting can also cause more false positives.

 

[Nightwalker]

Hell, Webroot has been doing that sort of thing for years (over a decade) with their file monitoring and their cloud Ai\machine learning and still can't manage to keep systems clean. Their guidance is that it will take up to 4 hours for a system to be rolled back. In testing I have seen systems at hour +144 not rolled back. You report that kind of stuff and I don't know where the report goes - the abyss - but a reply never, ever, comes back.

I don't know how many times, before I went to work for Blue Ridge Networks, I carefully gathered files, logs, hashes, and whatever else was requested, documented it and submitted the stuff. All I know, in the end, is that detection by hash was implemented.

It is a insult to compare Webroot Kool Aid technology with Kaspersky; Webroot can be bypassed with simple hollow Process while Kaspersky System Watcher has a much more refined and effective approach.

About the cloud Ai/machine learning, Kaspersky is much better too, serious, there arent on the same league, personally I would never use/trust/recommend a antivirus solution that doesnt have a proper emulator.


The Importance of the Anti Virus Emulator | Nota Bene: Eugene Kaspersky's Official Blog

https://media.kaspersky.com/pdf/b2b/machine-learning-and-human-expertize.pdf

 

[Sunshine-boy]

PPl you missed Dr.web! dr.web don't use Ai but its great

Virus analysts are not magicians and cannot instantly process the thousands and thousands of suspicious files received daily. Long gone are the times when anti-viruses could catch malware using only relevant virus signatures (i.e., records in virus databases) — i.e. detect only known viruses. If this were so till now, an anti-virus would be helpless in the face of unknown threats. However, an anti-virus remains the best and the only effective protection tool against all types of malicious threats — and, most importantly, — against viruses both known and unknown to the virus database.

Dr.Web incorporates many effective non-signature technologies for detecting and removing unknown malware. Together, they make it possible to detect the latest (unknown) threats before they are registered in the virus database. We'll describe just a few of them.

• Fly-Code technology ensures the high-quality scanning of packed executables and virtualized file execution to unpack any (even non-standard) packers; this makes it possible to detect viruses that are even unknown to Dr.Web anti-virus software.
• Origins Tracing treats a scanned executable as a specific sample which it then compares against the database of known malicious programs. The technology makes it highly likely that viruses not yet added to the Dr.Web virus database will be detected.
• Structural entropy analysis detects unknown threats by arranging pieces of code in objects protected with encryption compression, interrupting the routines they use, and utilizing some additional parameters. This allows Dr.Web to detect a substantial portion of unknown threats.
• ScriptHeuristic prevents any malicious browser scripts and PDF documents from being executed without disabling features provided by legitimate scripts. It protects against infection with unknown viruses that try to get into a system via a web browser. It works independently of the Dr.Web virus databases in any web browser.
• The traditional heuristic analyzer features routines to detect unknown malware. The heuristic analyzer relies upon knowledge (heuristics) about certain properties typical to virus code and, vice versa, those that are extremely rare in viruses. Each of these attributes is characterized by its “weight”— — that is to say, by a number whose module refers to the importance and severity of the attribute; and its sign, respectively, indicates whether that attribute confirms or refutes the hypothesis on the possible existence of an unknown virus in the code being analyzed.
• An execution emulator module is used to detect polymorphic and highly encrypted viruses when the search against checksums cannot be applied directly or is very difficult to perform (because secure signatures cannot be built). The method involves simulating the execution of an analyzed code by an emulator — a programming model of the processor (and, in part, PC, and OS).

 

[Nightwalker]

The things we are discussing here - like Ai and Machine Learning, were the promises that were made 20 years ago. Security fundamentals have always worked. Stick to the basics. The basics will take you further into the future than anything else. 1 + 1 = 2. If you block it from executing in the first place, then you do not have deal with any unpleasant consequences afterwards. Ai, Machine Learning, behavioral analysis, etc - they require you to execute something. If they don't catch it, you're beat ! You're much better off blocking by default. Default deny will save you much pain. So plainly obvious and common sense, you have to wonder why so very few people actually adhere to it.

Because it isnt pratical or viable at all.

Default deny is great for enterprise environments that should have strictly controlled software usage pattern, but it doesnt make any sense for the average home user and I will explain why:

The worker should not open "free premium netflix generator.exe" or "Document.pdf.scr" at his job, but at his home is a very different scenario that default deny simple doesnt work; he wants to open "Photoshop keygen.exe" and he will find a way to run it unless that his antivirus tell him explicitly that it isnt safe to do so.

I will never consider default deny a security solution at all for home (and my) usage because if someone clicks to open a file is that they want to run it and they want to know that its safe to do so, thats why default deny isnt mainstream.

Companies like Kaspersky, Symantec, ESET could make default deny solutions in a very short time and this would greatly decrease their R&D expenses, but would it be feasible?

There is no silver bullet, if there was something better for mainstream usage besides antivirus I would bet all my money that the leaders of this market would already be using it (antivirus is expensive to develop and maintain).

 

[bribon77]

Because it isnt pratical or viable at all.

Default deny is great for enterprise environments that should have strictly controlled software usage pattern, but it doesnt make any sense for the average home user and I will explain why:

The worker should not open "free premium netflix generator.exe" or "Document.pdf.scr" at his job, but at his home is a very different scenario that default deny simple doesnt work; he wants to open "Photoshop keygen.exe" and he will find a way to run it unless that his antivirus tell him explicitly that it isnt safe to do so.

I will never consider default deny a security solution at all for home (and my) usage because if someone clicks to open a file is that they want to run it and they want to know that its safe to do so, thats why default deny isnt mainstream.

Companies like Kaspersky, Symantec, ESET could make default deny solutions in a very short time and this would greatly decrease their R&D expenses, but would it be feasible?

There is no silver bullet, if there was something better for mainstream usage besides antivirus I would bet all my money that the leaders of this market would already be using it (antivirus is expensive to develop and maintain).

According to which users. If you are an advanced user If it is important to Default deny . if it is for normal user I agree with you. why normal user does not know more.

 

[509322]

Because it isnt pratical or viable at all.

Default deny is great for enterprise environments that should have strictly controlled software usage pattern, but it doesnt make any sense for the average home user and I will explain why:

The worker should not open "free premium netflix generator.exe" or "Document.pdf.scr" at his job, but at his home is a very different scenario that default deny simple doesnt work; he wants to open "Photoshop keygen.exe" and he will find a way to run it unless that his antivirus tell him explicitly that it isnt safe to do so.

I will never consider default deny a security solution at all for home (and my) usage because if someone clicks to open a file is that they want to run it and they want to know that its safe to do so, thats why default deny isnt mainstream.

Companies like Kaspersky, Symantec, ESET could make default deny solutions in a very short time and this would greatly decrease their R&D expenses, but would it be feasible?

There is no silver bullet, if there was something better for mainstream usage besides antivirus I would bet all my money that the leaders of this market would already be using it (antivirus is expensive to develop and maintain).

Like I said, "Users want to use stuff" and that is why there is default allow. It is because of that mentality that 3/4 of the world gets infected.

What is true of the Enterprise worker is 100X more true of the home user. The home user does not know what they are doing, the Antivirus is not good enough to tell them what to do in 100 % of cases 100 % of the time. You guys that want to engage in high-risk computing but still expect antivirus to save your asses are clinging to a false hope and have been doing so for decades. The AV industry has been telling you that for at least the past 5 years.

But, if the home user isn't a high-risk computer-using negligent knucklehead, most AVs are more than sufficient in the high 90 percentiles for typical, humdrum, grandma, uncle Bob day-to-day computing. When you get into Keygens - that isn't typical computing - you enter into negligent, high-risk computing and push the boundaries of AV lab results. The only ones who get bent out of shape about an AV only performing at 95 % or 96 % are the ultra-paranoid and high-risk user types who summarily pronounce that AV as garbage - which is the whole premise of this thread "Do you really understand AV test results ?"

Kaspersky, Symantec and ESET do make default deny features, modes or settings within their products. The user just has to enable and\or configure those features, modes or settings - but the claim that they do not make default-deny is correct only in that they do not make a standalone dedicated default deny product.

I remember the one guy who was miffed because he downloaded a PDF and scanned it with ESET, executed the PDF and his system was encrypted along with all his school work files - without having any backups. You tell me, whose fault is all of that - ESET's or the user's ? The user wants to blame it all on ESET. The ESET EULA clearly states that it is the user's fault - all responsibility rests with the user for what happens on the system. Everything that happens on a system starts with the user and not using the ESET HIPS. If he had configured the ESET HIPS, then the ransomware would have triggered some form of HIPS action.

Locking people out of the system along with a security hardened OS and programs isn't a magic silver bullet, but it is extremely high security. It's a better protection model than anything anyone has been able to come up with yet.

Like I said, use what you like. There is enough security softs to accommodate everyone.

PS - use of keygens is piracy. AppGuard LLC is absolutely against any and all software piracy.

 

[Rebsat]

these are what I did

Spoiler: tweaks

1/ Process Lasso: disallowed wscript, cscript, powershell.exe, powershell_ise.exe, java.exe, javaw.exe
2/ Group Policy (SRP): blocked some extensions: .hta, .jar, .scr
3/ Regedit: blocked windows script host
4/ Windows Firewall:
- blocked all inbound connections
- block outbound: msra.exe, msha.exe, wscript, cscript, powershell, powershell_ise, conhost, cmd

when I need to execute a script, I just need to unblock WSH via registry and wscript, cscript from process lasso

I make sure I only execute .exe files so avast's hardened mode will work

I think avast + these tweaks, avast can be more powerful than Emsisoft but still a bit behind kaspersky IS (fully tweaked). If we trust HM 100% because it's default-deny and can block safe programs. The same programs might be whitelisted automatically after a few days when they have enough number of users

One more question bro. What about CFW? Will it protect my system from those scripts and scriptlets using your custom tweaks/settings at the same level of Emsisoft and Kaspersky? Thanks

 

[Evjl's Rain]

One more question bro. What about CFW? Will it protect my system from those scripts and scriptlets using your custom tweaks/settings at the same level of Emsisoft and Kaspersky? Thanks

it will protect even better CF is somewhat like a default-deny solution while AVs are blackilisting solution
you can use cruelsister's configuration to have ~99.9-100% effectiveness. Almost better than anything else

you can also add this to CS's CF settings to strengthen protection against scripts and vulnerable extensions (only read autosandbox, ignore firewall settings)
SECURE - Evjl's Rain's security config

 

[shmu26]

Because it isnt pratical or viable at all.

Default deny is great for enterprise environments that should have strictly controlled software usage pattern, but it doesnt make any sense for the average home user and I will explain why:

The worker should not open "free premium netflix generator.exe" or "Document.pdf.scr" at his job, but at his home is a very different scenario that default deny simple doesnt work; he wants to open "Photoshop keygen.exe" and he will find a way to run it unless that his antivirus tell him explicitly that it isnt safe to do so.

I will never consider default deny a security solution at all for home (and my) usage because if someone clicks to open a file is that they want to run it and they want to know that its safe to do so, thats why default deny isnt mainstream.

Companies like Kaspersky, Symantec, ESET could make default deny solutions in a very short time and this would greatly decrease their R&D expenses, but would it be feasible?

There is no silver bullet, if there was something better for mainstream usage besides antivirus I would bet all my money that the leaders of this market would already be using it (antivirus is expensive to develop and maintain).

Voodooshield is striving to give the average home user a default/deny setup that is simple enough to just work.

 

[shmu26]

it will protect even better CF is somewhat like a default-deny solution while AVs are blackilisting solution
you can use cruelsister's configuration to have ~99.9-100% effectiveness. Almost better than anything else

you can also add this to CS's CF settings to strengthen protection against scripts and vulnerable extensions (only read autosandbox, ignore firewall settings)
SECURE - Evjl's Rain's security config

You can greatly increase Comodo script protection by going into advanced settings/miscellaneous/heuristic command line, and turn on full protection for the whole vulnerable process list. You can also add to the list, if you wish. It's pretty flexible.

 

[Andy Ful]

Like I said, "Users want to use stuff" and that is why there is default allow. It is because of that mentality that 3/4 of the world gets infected.

What is true of the Enterprise worker is 100X more true of the home user. The home user does not know what they are doing, the Antivirus is not good enough to tell them what to do in 100 % of cases 100 % of the time. You guys that want to engage in high-risk computing but still expect antivirus to save your asses are clinging to a false hope and have been doing so for decades. The AV industry has been telling you that for at least the past 5 years.

But, if the home user isn't a high-risk computer-using negligent knucklehead, most AVs are more than sufficient in the high 90 percentiles for typical, humdrum, grandma, uncle Bob day-to-day computing. When you get into Keygens - that isn't typical computing - you enter into negligent, high-risk computing and push the boundaries of AV lab results. The only ones who get bent out of shape about an AV only performing at 95 % or 96 % are the ultra-paranoid and high-risk user types who summarily pronounce that AV as garbage - which is the whole premise of this thread "Do you really understand AV test results ?"

Kaspersky, Symantec and ESET do make default deny features, modes or settings within their products. The user just has to enable and\or configure those features, modes or settings - but the claim that they do not make default-deny is correct only in that they do not make a standalone dedicated default deny product.

I remember the one guy who was miffed because he downloaded a PDF and scanned it with ESET, executed the PDF and his system was encrypted along with all his school work files - without having any backups. You tell me, whose fault is all of that - ESET's or the user's ? The user wants to blame it all on ESET. The ESET EULA clearly states that it is the user's fault - all responsibility rests with the user for what happens on the system. Everything that happens on a system starts with the user and not using the ESET HIPS. If he had configured the ESET HIPS, then the ransomware would have triggered some form of HIPS action.

Locking people out of the system along with a security hardened OS and programs isn't a magic silver bullet, but it is extremely high security. It's a better protection model than anything anyone has been able to come up with yet.

Like I said, use what you like. There is enough security softs to accommodate everyone.

PS - use of keygens is piracy. AppGuard LLC is absolutely against any and all software piracy.

Thanks. The above post is really excellent, and worth reading twice.
My thought about malware is that people created the parallel world that reflects the human nature and the real world. In the real world, we have to share the space with viruses (even in our body). In the parallel world, the computer networks have to share the space with malware (even in computers). And, I think that there is no way out, and nothing can change this.
Default-deny protection simply reflects the strict hygiene (high above average, like surgeons do). Some people like it, but most not. And some people even do not wash their hands before meals (run keygens and ignore AV alerts). So, let's get accustomed to viruses, some of them (AVs) are already on our computers, and there will be more, soon.​

 

[bribon77]

The problem with home users is that they do not like security or that they are not interested. They also do not know that Voodooshield exists, nor Hard_Configurator,AppGuard or exe radar. nor Comodo Firewall with the configuration of Cs ... I did not know either until I met MT.

 

[Andy Ful]

The problem with home users is that they do not like security or that they are not interested. They also do not know that Voodooshield exists, nor Hard_Configurator,AppGuard or exe radar. nor Comodo Firewall with the configuration of Cs ... I did not know either until I met MT.

Some even do not know that they have an antivirus, and most know the word firewall from films (if any).

 

[509322]

Thanks. The above post is really excellent, and worth reading twice.
My thought about malware is that people created the parallel world that reflects the human nature and the real world. In the real world, we have to share the space with viruses (even in our body). In the parallel world, the computer networks have to share the space with malware (even in computers). And, I think that there is no way out, and nothing can change this.
Default-deny protection simply reflects the strict hygiene (high above average, like surgeons do). Some people like it, but most not. And some people even do not wash their hands before meals (run keygens and ignore AV alerts). So, let's get accustomed to viruses, some of them (AVs) are already on our computers, and there will be more, soon.​

In the future (not next year, or even the next decade, but decades from now), security will be much different. Platforms and devices will be restricted use - default-deny right off the manufacturing floor. Users aren't going to be able to do what they want - they won't be able to install stuff; they're going to be heavily restricted - if not locked out and held responsible. It's the only answer as the ecosystem and attack surface is growing exponentially at a exponentially monstrous rate.

That's right... the digital police are coming. Oh yes, they most certainly are - breaking down the doors of users who cause network infections and such through their own ignorance, and ummm, haling them off to the klink. This is the future. I know its true because I saw variations of it on TV in at least 10 or more movies. This digital-Orwellian future is in the mail - it is coming. And users are going to go to jail in the future for stuff they do today without thought.

 

[mlnevese]

The average computer user wants to turn on their computers and do some work, watch some videos, see some some pictures, visit some web sites and that's it. They do not want to know how it works they just want to use it and be done with it, and you know what... they are not wrong.

I don't know how every single piece of my car works. In the morning I just turn the key and expect it to turn on, ignite the engine and start moving.If something is wrong with it that is not obvious or easy to identify I take it to a specialist.

Expecting the average user to take all the time and study it takes to understand how the OS and security programs work is unrealistic. It's not their profession, it's not their hobby, it's just a tool they use. That's why I believe security software should be as automatic as possible. It doesn't mean it shouldn't have more advanced tools and configurations for advanced users or IT professionals but it DOES mean that any security product should offer as much protection out of the box as possible.

 

[509322]

Some even do not know that they have an antivirus, and most know the word firewall from films (if any).

Microsoft needs to start educating people about its high-risk, general OS. Microsoft created the ecosystem, they created the risk, they keep the Windows sales engine purring along, they reap the huge profits, they and the OEMs need to educate the IT security illiterate average Joes. Or just maybe everybody that doesn't know enough about security should just move to Chromebook. They'd be better off.

Governments should impose "average Joe education" with decent minimum standards and not let them raise prices because they have to produce teaching materials. They're all making a lot of money. The money is there in their budgets or it can be made available.

 

[509322]

The average computer user wants to turn on their computers and do some work, watch some videos, see some some pictures, visit some web sites and that's it. They do not want to know how it works they just want to use it and be done with it, and you know what... they are not wrong.

I don't know how every single piece of my car works. In the morning I just turn the key and expect it to turn on, ignite the engine and start moving.If something is wrong with it that is not obvious or easy to identify I take it to a specialist.

Expecting the average user to take all the time and study it takes to understand how the OS and security programs work is unrealistic. It's not their profession, it's not their hobby, it's just a tool they use. That's why I believe security software should be as automatic as possible. It doesn't mean it shouldn't have more advanced tools and configurations for advanced users or IT professionals but it DOES mean that any security product should offer as much protection out of the box as possible.

I could make a long reply to this, but you have average Joes out there that do not even know that Windows Defender ships with Windows.

The amount of education effort required to get a decent handle on Windows, general security, and security softs is not that onerous.

You cannot rely upon a security soft alone. It takes more than that nowadays. If a person does not understand this fact or is ignorant of it, then they are at a great disadvantage in securing their systems. What you don't know can and will hurt you.