Accusations/comments on Virus.gr tests

The antivirus tests of virus.gr have been accused too many times already during the last years and, after reading an article by Mr. Clementi recently (http://www.av-comparatives.org/forum/index.php?page=Thread&threadID=753 - Andreas says that they are not all coming from him or reflecting his opinion), i feel the need to say this:

a) Not all products are tested equally:

- some products are trial versions, some others are full versions

This is false, all software have FULL capabilities (unless mentioned by virus.gr) during the antivirus test

- some use best settings, some others use "not default settings" (whatever this means); this may be due the trial versions limitations

This is false, all software have the best scanning options to achieve maximum detection

- not all products are updated at the same time/day/week (but this seems now to be solved in the last tests)

This is false, all products are updated at the same day with a few hours difference; read the notes of the latest test

b) Samples:

- samples were choosen by using those products: Kaspersky, F-Prot, Nod32, Dr.Web, BitDefender and McAfee; requirement: unique name. This leads to a major flaw toward KAV and products which use many unique names instead of generic names.

This is false, the test-bed is chosen using more than one AV so is it impossible for the test results to be in favor of one AV

- sample source are mainly other VXer: as VXer use mainly KAV to share their samples, the tests and samples are flawed in favor of KAV.

This is false, the test-bed is chosen using more than one AV so is it impossible for the test results to be in favor of one AV

- there is a huge amount of garbage included in the test-set, even if he says he removed "most" of it.

Most of the samples are gathered from in-the-wild sources and are in fact tested one-by-one, so they are basically among the samples sent by emails to common users or downloaded by “suspicious” sites that promote warez or other illegal software/multimedia; samples are not selected but collected. Nevertheless, IF this accusation was true, it would also make other comparative tests (which also make false positive tests as well as on-demand tests) innacurate and untrustworthy, since there are MANY AV software besides Kaspersky that detect ALMOST all of this "huge garbage collection" as viruses; Now, wouldn't this make all these software also untrustworthy, since they detect so many thousands of "garbage files" as viruses?... Think about it...

- at least some vendors got from them their whole collection in past, as some vendors were able to determine how much garbage is in it (even if some users in forums say that virus.gr do not sends any sample to av vendors)

This is completely false. No av vendor has got the collection owned by virus.gr (at least I am unaware of such a thing). So, I do not see how this was achieved, if so.

- the test-set contains lot of DOS files

The DOS (MS-DOS) files will be excluded in future test, this was my intention since the last test anyway.

c) Miscellanous:

- the Virus.gr site is not a team of people, it is atm 1 person. The tester behind it is a viruscollector (VXer) with very limited programming skills or coding skills in general (which means he can not check for garbage in the test-set, etc.).

This is true, but fact is that virus.gr is assisted by other people when it comes to testing samples for av tests.

The guy seems to have genuine intentions and tries to give his best while doing his tests, but the tests need still improvements and better methods. In any case, the results may be interesting to look at, as they include also some minor AV products which are usually not tested.

This is also true, I am not a professional but this does not mean that the results of the test do not reflect the detection ratio of the antivirus software tested.

Either way, I believe that each test has a different methodology and it is pointless to say that one test is correct and another one isn’t. So, I would suggest that if you do not agree with the methodology used by this site, just visit other sites that also perform antivirus tests. We do not sell neither promote antivirus software; this is not the case for virus.gr.

But if you DO have some comments on virus.gr tests and disagree on the test results, “be a gentleman” and post your question on our forum section instead of making accusations without evidence or without letting us state OUR POINT OF VIEW. This is NOT ETHICAL.