Video The Shadowra's Battlefield Antivirus 2021

Source
https://www.youtube.com/watch?v=LewkG7sUNnQ
Video created by
Shadowra

Andy Ful

Level 80
Verified
Helper
Top poster
Developer
Well-known
Dec 23, 2014
6,994
The problem of corrupted files is more complex. These files can be a part of the larger attack (payloads) but simply require the initial loader or external resources (*.tmp, *.dll, *.dat, etc.).
They cannot be executed alone, so they cannot be also detected dynamically in such tests. They can be detected when the attack is monitored starting from the initial malware and ending with the final payload. After detecting the full attack, the signatures of such payloads can be blacklisted as for standard malicious files.

Post edited.
 
Last edited:

Andy Ful

Level 80
Verified
Helper
Top poster
Developer
Well-known
Dec 23, 2014
6,994
I was thinking about how this test can be improved. My final thought is that it should be repeated several times. This can help to eliminate some random factors like this one mentioned by me in the previous post.

The test can be divided into two parts because the phishing part is independent of the rest.

It could also help if @Shadowra could show more Indicators of Compromise (IoCs) because it is often hard to see if the leftovers are real IoCs or not.

The method of calculating the scorings on the basis of on-demand scanners seems to show which AV is cleaner (leaves a smaller number of leftovers) and this can be related to the architecture of detections. Some AVs rely on the fast signatures in the cloud and these AVs will detect most malware without executing them (small number of leftovers). Others can rely more on post-execution detection (more leftovers).
 

harlan4096

Moderator
Verified
Staff member
Malware Hunter
Well-known
Apr 28, 2015
7,978
Read previous comments about the test and the samples used (many corrupted files, signed and/or old "legit" PUP), those results can lead to false conclusions.

It is not clear in that list of punctuation which systems remained protected.

I would also like to see there Kaspersky Removal Tool running as Second Opinion Scanner.
 
F

ForgottenSeer 92963

Antivirus tested : Avast, Avira, Microsoft Defender, WiseVector, Bitdefender, Kaspersky, ESET, F-Secure, Trend Micro, Norton.

Shooting time: 10 hours
Editing time : 1 hour
Upload : 6 minutes
TWO FLAWS IN THIS TEST:
  1. DID YOU CHECK THE SAMPLES WERE REALLY MALWARE AND EXECUTED CORRECTLY? THIS COULD EXPLAIN THE CORRUPTED FILES. :oops:

  2. TESTED SEQUENTIALLY (SHOOTING TIME 10 HOURS), NOT IN PARALLEL MIGHT INFLUENCE RESULTS WITH AV's SHARING DETECTIONS (e.g. VT). :unsure:

QUESTION: ARE THE AV's WITH THE BEST DETECTIONS TESTED LAST? IF SO, YOU HAVE PROVEN THAT SHARING SAMPLES WORKS. :)

I really applaud your time and effort you put into these tests. Also even when you buy additional 10 pc's and test AV's at the same time and run the malware samples in a sandbox to check they are not corrupted and really infect the system, you must make sure your ISP and DNS don't have any kind of malware detection or url monitoring in their 'extended/additional' service offering because this could affect bad URL detection also. I feel a bit sorry for you, all that time and effort you put in this test. Enjoy your holiday. (y)
 
Last edited by a moderator:

Shadowra

Level 20
Thread author
Verified
Malware Tester
Sep 2, 2021
956
TWO FLAWS IN THIS TEST:
  1. DID YOU CHECK THE SAMPLES WERE REALLY MALWARE AND EXECUTED CORRECTLY? THIS COULD EXPLAIN THE CORRUPTED FILES. :oops:

  2. TESTED SEQUENTIALLY (SHOOTING TIME 10 HOURS), NOT IN PARALLEL MIGHT INFLUENCE RESULTS WITH AV's SHARING DETECTIONS (e.g. VT). :unsure:

QUESTION: ARE THE AV's WITH THE BEST DETECTIONS TESTED LAST? IF SO, YOU HAVE PROVEN THAT SHARING SAMPLES WORKS. :)

1. I don't always check, basically I trust my supplier, especially since this battle was long....

2. I think it has nothing to do, during the comparison, the antiviruses are tested fairly, default, and PUPs activated.
I will not prioritize any software :)

3. No. The last ones were Bitdefender and Norton which was tested.
 
F

ForgottenSeer 92963

1. I don't always check, basically I trust my supplier, especially since this battle was long....

2. I think it has nothing to do, during the comparison, the antiviruses are tested fairly, default, and PUPs activated.
I will not prioritize any software :)

3. No. The last ones were Bitdefender and Norton which was tested.

1. That is a problem

2. Yes it has, when they are not tested at the same time, the playing field is not equal

3. It makes a difference. which was the testing sequence and what were the results in your scoring method
 

Shadowra

Level 20
Thread author
Verified
Malware Tester
Sep 2, 2021
956
1. That is a problem

2. Yes it has, when they are not tested at the same time, the playing field is not equal

3. It makes a difference. which was the testing sequence and what were the results in your scoring method

Not at all.
You should know that during the tests, I do not send ANYTHING to the editors!
It's only after the video is published that I send them, so as not to interfere.
By the way, I don't use VirusTotal to check if a file is malicious :) but a site that does not distribute

EDIT :

3.
I would have liked to send you the note file, but I'm not at home ;) (even if me and my mother have the same speedtest ^^ )

I note:

- User interface (user-friendly? messy? complicated?)
- Phishing reactivity
- Reactivity malicious link
- Analysis (time, score etc)
- Cleaning (time, how many it removes)
- Launch reaction (what Avira, Trend, F-Secure and Microsoft Defender did)

I hope I answered your question. If you want to talk about specific points, don't hesitate.
 
F

ForgottenSeer 92963

@Shadowra what was the sequence in which you tested the AV's

AV's share their new detections. Usually when an A-lister detects something on VirusTotal, within 12 hours all A-listers will detect that samplev(by hash and URL). Besides thar AV's using an other Engine (e.g Bitdefender or Avast) will get the updates through inter-company channels.
 
Last edited by a moderator:

Shadowra

Level 20
Thread author
Verified
Malware Tester
Sep 2, 2021
956
@Shadowra what was the sequence in which you tested the AV's

Here is the order

  • Avast
  • Avira
  • Microsoft Defender
  • Eset
  • F-Secure
  • Kaspersky
  • WiseVector
  • Trend Micro
  • Bitdefender
  • Norton

Then the test protocol is identical

AV's share their new detections. Usually when an A-lister detects something on VirusTotal, within 12 hours all A-listers will detect that samplev(by hash and URL). Besides thar AV's using an other Engine (e.g Bitdefender or Avast) will get the updates through inter-company channels.

=> By the way, I don't use VirusTotal to check if a file is malicious :) but a site that does not distribute
 
Last edited:
F

ForgottenSeer 92963

=> By the way, I don't use VirusTotal to check if a file is malicious :) but a site that does not distribute
I am not posting that you are doing it, but the AV's share new detections (inter company and through VT), usually with some delay (on average 12 hours)
 
  • Like
Reactions: Sorrento

NewbyUser

Level 2
Verified
Well-known
Jul 16, 2021
52
I am not posting that you are doing it, but the AV's share new detections (inter company and through VT), usually with some delay (on average 12 hours)
AV's don't share detections through VT, VT is a private entity run by Google. The AV companies also do not typically share with each other. All major AVs have excellent collecting, coding and distribution of signatures most of this process is automated and takes only a few hours. Each AV is a privately held/run company with no incentive to share detections or coding mechanisms, that would put themselves out of business.

Some companies don't produce an engine of their own and license from another company. This may be what you are confusing with "sharing" detections.
 

SecureKongo

Level 29
Verified
Top poster
Well-known
Feb 25, 2017
1,835
Too much criticism and not enough appreciation for all the effort he has made creating the video. It's not even possible for most testers to test the AVs simultaniously if they don't have a NASA PC. Only proper option would be creating an individual VM for each AV and I am pretty sure that with such a big list of AVs pretty much every computers hardware would struggle out there. Keep in mind that he is a hobby-tester after all. @Kees1958
 
Last edited:

NewbyUser

Level 2
Verified
Well-known
Jul 16, 2021
52
Too much criticism and not enough appriciation for all the effort he has made creating the video. It's not even possible for most testers to test the AVs simultaniously if they don't have a NASA PC. Only proper option would be creating an individual VM for each AV and I am pretty sure that with such a big list of AVs pretty much every computers hardware would struggle out there. Keep in mind that he is a hobby-tester after all. @Kees1958
Excellent Point!! Didn't mean to add to the distractions
 

Shadowra

Level 20
Thread author
Verified
Malware Tester
Sep 2, 2021
956
I really applaud your time and effort you put into these tests. Also even when you buy additional 10 pc's and test AV's at the same time and run the malware samples in a sandbox to check they are not corrupted and really infect the system, you must make sure your ISP and DNS don't have any kind of malware detection or url monitoring in their 'extended/additional' service offering because this could affect bad URL detection also. I feel a bit sorry for you, all that time and effort you put in this test. Enjoy your holiday. (y)

Why be sorry?
A forum is made to debate, there was no amiosite towards me :)
You didn't disrespect me or anything, you had questions, I answered :)

To answer you with the DNS of my operator, no filtering Phishing or malware is proposed.
They just offer an Antivirus (McAfee).
I don't use the DNS of the fai, but NextDNS.
And the web filtering of NordVPN is disabled.

Have a great holiday. 😉
 
F

ForgottenSeer 92963

Too much criticism and not enough appreciation for all the effort he has made creating the video
I posted this appreciation: "I really applaud your time and effort you put into these tests" in post #29 :unsure: not enough? then I will say it again (and more explicitly)

@Shadowra "I really appreciate (and applaud) the time and effort you put into these tests".
 
Last edited by a moderator: