- Apr 9, 2018
- 211
Dear Readers!
With this summary we conclude this year’s penultimate series of Advanced In-The-Wild Malware Tests. As of September 2024, we have made a few changes under the hood in the Windows 10/11 security software testing application.
The first and biggest is that we have developed a new method of capturing more evidence of malware sample detection during the test in the form of screenshots taken several times a minute.
We use an OCR tool to read text from the images. Based on this text, we compare the keywords with matching alerts for the anti-virus software in question. If there is a positive match, we store all the records in a database, from which summaries are generated for the vendors. This image recognition capability also allows us to more thoroughly analyse potential malware samples before qualifying them for testing – we reject installers, unwanted (non-malicious) applications, corrupted files and other files that cannot be run in a Windows 11 environment:
As an example - a "system error" is reported for a potentially malicious keylog.exe application:
During the initial selection, the corrupted malware sample is rejected due to the detection of a Windows error with the keywords "system error":
Evidence of sample removal from the assay at the preliminary analysis stage:
In summary, the OCR tool we have incorporated into the testing process is used to capture anti-virus alerts and to more accurately identify potential malware samples before they are qualified for testing. The screenshots provide the vendors with further and irrefutable evidence of whether or not malware has been detected.
The OCR tool is an additional opinion from the test, alongside the Sysmon logs and the logs generated by the security software under test. See our methodology page for more details.
We have added optional changed onto the backend as well. You can read them all on the transparency website: Changelog » AVLab Cybersecurity Foundation
September 2024
Results: Recent Results » AVLab Cybersecurity Foundation
And the publication: Summary Of The Advanced In-The-Wild Malware Test - September 2024 » AVLab Cybersecurity Foundation
We are also working with other vendors to add them to the tests. This is not always technically easy, so we ask for your understanding.
I hope that these changes will contribute to even better confidence in us and in our tests.
With this summary we conclude this year’s penultimate series of Advanced In-The-Wild Malware Tests. As of September 2024, we have made a few changes under the hood in the Windows 10/11 security software testing application.
The first and biggest is that we have developed a new method of capturing more evidence of malware sample detection during the test in the form of screenshots taken several times a minute.
We use an OCR tool to read text from the images. Based on this text, we compare the keywords with matching alerts for the anti-virus software in question. If there is a positive match, we store all the records in a database, from which summaries are generated for the vendors. This image recognition capability also allows us to more thoroughly analyse potential malware samples before qualifying them for testing – we reject installers, unwanted (non-malicious) applications, corrupted files and other files that cannot be run in a Windows 11 environment:
As an example - a "system error" is reported for a potentially malicious keylog.exe application:
During the initial selection, the corrupted malware sample is rejected due to the detection of a Windows error with the keywords "system error":
Evidence of sample removal from the assay at the preliminary analysis stage:
In summary, the OCR tool we have incorporated into the testing process is used to capture anti-virus alerts and to more accurately identify potential malware samples before they are qualified for testing. The screenshots provide the vendors with further and irrefutable evidence of whether or not malware has been detected.
The OCR tool is an additional opinion from the test, alongside the Sysmon logs and the logs generated by the security software under test. See our methodology page for more details.
Security alerts example screenshots:
We have added optional changed onto the backend as well. You can read them all on the transparency website: Changelog » AVLab Cybersecurity Foundation
September 2024
Results: Recent Results » AVLab Cybersecurity Foundation
And the publication: Summary Of The Advanced In-The-Wild Malware Test - September 2024 » AVLab Cybersecurity Foundation
We are also working with other vendors to add them to the tests. This is not always technically easy, so we ask for your understanding.
I hope that these changes will contribute to even better confidence in us and in our tests.
