I do have a wild theory...
...about where those false alarms might have their origin!
Everything started with me scanning the MobMapUpdater v1.75 (the one which is now marked as "trojan-infected" by Kaspersky) with VirusTotal, a nice website that scans a file with many anti-virus engines. I wanted to make sure that the file does not raise any false alarms before I release it - and it didn't. Or better said: it didn't raise any alarm from Kaspersky or AntiVir. There was actually one red entry in the list: Prevx - which does seem to be some kind of heuristic malware scanner - classified the Updater as "Malicious Software" and provided some additional information about the file. Well, I chose to ignore this false alarm, as I have not found any possibility until today to inform the company that creates Prevx about the false alarm, and because Prevx isn't really a widespread malware scanner. Kaspersky and AntiVir were silent, that was important to me. So I released v1.75.
12 hours later, Kaspersky has "invented" the new WoW "trojan" Trojan-PSW.Win32.WOW.bcm (notice the naming similarities to the other suspected "trojans" in older versions of the MobMapUpdater). Well, actually it didn't take them 12 hours to create this new "trojan", more like 6 hours and some additional hours for the definition update, as Kaspersky's Virus Watch indicates. The signature of this "trojan" was probably created right from the new MobMapUpdater executable, which is why the MobMapUpdater triggers the alarm.
The big question is: what caused Kaspersky to classify the MobMapUpdater as a trojan (the fact that it is not a trojan is proven by Kaspersky's virus analysts themselves, as they have manually analyzed the MobMapUpdater twice and found it to be entirely clean and trojan-free, as well as the virus analysts at Avira) and where do they always get their "samples" from? I do have a theory, and that theory includes VirusTotal and the Prevx heuristic scanner I mentioned earlier.
As you can read in this blog post in the VirusTotal blog, VirusTotal does silently distribute uploaded files to all the anti-virus companies that provide theirs scanning engines to the site. I don't know what criteria must be met for the site to distribute a sample, but I suspect that if just one of the scanners believes to find something, a file is being distributed as "potentially infected" to the antivirus companies. And that one single scanner might have been Prevx in the MobMapUpdater case!
So I took a look at the Prevx report that the newest MobMapUpdater v1.76 (the one I compiled in response to the Kaspersky false alarm; look at the blog post below for details) was generating, and found an interesting sentence in there. It says:
What? E-Mail? The MobMapUpdater doesn't send any E-Mail! But...it says "MIME Email". And there it became clear why the Prevx heuristic algorithm might believe that the MobMapUpdater does send E-Mail. Of course it doesn't do that, but it does "send" something: it does upload your collected data to the MobMap website, if you wish so. And to do this, it uses an HTTP POST request with MIME encoding for the actual data - and the code component which does that seemed to be enough to trigger the heuristic algorithm which believed that the MIME encoding stuff in the MobMapUpdater just has to be used to send E-Mail (deliberately ignoring that you can do many other things with the MIME encoding standard, like for example upload data to a website).
And if this Prevx heuristics crap can make such a false assumption, the heuristic algorithms at Kaspersky which they probably use to automatically classify new samples (which they get - for example - from the VirusTotal website!) and automatically generate signatures for these samples if the heuristic does believe to have found something suspicious might make the same mistake!
Most people don't really think about how those hundreds of thousands of records in the signature files of all those antivirus scanners are actually created (Kaspersky for example says that they currently have 839572 single records in their database), and if someone actually does think about it, he usually believes that there are many professional virus analysts somewhere in a lab analyzing every single suspected file by hand and crafting signatures to detect them. That may have been true 10 years ago, but today most of the "viruses" and "trojans" in the antivirus databases have never been analyzed by any human being.
Instead, heuristic algorithms try to determine what a suspected file might be doing on a system, and depending on that analysis a file is being classified as containing a virus or not. If such a heuristic finds a new virus which there exists no signature for, it pulls up a new name (hence the naming similarities in the "trojans" mentioned earlier!) and automatically creates a signature for it. No human being involved here - at least not in the everyday process of analyzing new files.
That is how the antivirus companies are able to keep up with the thousands of malware variants that keep popping up every day without having to pay thousands of professional software analysts 24/7. And I think that is also the reason why the MobMapUpdater keeps raising false alarms at the Kaspersky scanner. The last two times, a sample was sent to Kaspersky for manual analysis, and the manual analysis by a skilled virus analyst always quickly led to the conclusion that the file is indeed not a trojan and doesn't contain a trojan, which was followed by Kaspersky removing the signature from their definition files. But the next MobMapUpdater version just generated a new signature through the automated process, and the whole crap started all over again...
So what could I do to prevent all this crap from happening? Make it clear to the heuristics that the MobMapUpdater does not "send MIME Email"! So I ripped out the MIME encoding component I used previously (which actually wasn't written by myself, but a component from a library offered by Borland) and replaced it by a self-written minimalistic MIME encoder that just does what I need to upload the collected data to the website. To be on the safe side I also replaced the Base64 encoder I used previously (also a stock component) with a self-written Base64 encoder. Then I ran the file through VirusTotal (and therefore the Prevx heuristic scanner): 0 alarms, the file is found to be perfectly clean. I hope that these changes suffice to keep the newest version of the MobMapUpdater (v1.77, online for download now!) from appearing in Kaspersky's definition files, so that this odyssey finally has an end!
Everything started with me scanning the MobMapUpdater v1.75 (the one which is now marked as "trojan-infected" by Kaspersky) with VirusTotal, a nice website that scans a file with many anti-virus engines. I wanted to make sure that the file does not raise any false alarms before I release it - and it didn't. Or better said: it didn't raise any alarm from Kaspersky or AntiVir. There was actually one red entry in the list: Prevx - which does seem to be some kind of heuristic malware scanner - classified the Updater as "Malicious Software" and provided some additional information about the file. Well, I chose to ignore this false alarm, as I have not found any possibility until today to inform the company that creates Prevx about the false alarm, and because Prevx isn't really a widespread malware scanner. Kaspersky and AntiVir were silent, that was important to me. So I released v1.75.
12 hours later, Kaspersky has "invented" the new WoW "trojan" Trojan-PSW.Win32.WOW.bcm (notice the naming similarities to the other suspected "trojans" in older versions of the MobMapUpdater). Well, actually it didn't take them 12 hours to create this new "trojan", more like 6 hours and some additional hours for the definition update, as Kaspersky's Virus Watch indicates. The signature of this "trojan" was probably created right from the new MobMapUpdater executable, which is why the MobMapUpdater triggers the alarm.
The big question is: what caused Kaspersky to classify the MobMapUpdater as a trojan (the fact that it is not a trojan is proven by Kaspersky's virus analysts themselves, as they have manually analyzed the MobMapUpdater twice and found it to be entirely clean and trojan-free, as well as the virus analysts at Avira) and where do they always get their "samples" from? I do have a theory, and that theory includes VirusTotal and the Prevx heuristic scanner I mentioned earlier.
As you can read in this blog post in the VirusTotal blog, VirusTotal does silently distribute uploaded files to all the anti-virus companies that provide theirs scanning engines to the site. I don't know what criteria must be met for the site to distribute a sample, but I suspect that if just one of the scanners believes to find something, a file is being distributed as "potentially infected" to the antivirus companies. And that one single scanner might have been Prevx in the MobMapUpdater case!
So I took a look at the Prevx report that the newest MobMapUpdater v1.76 (the one I compiled in response to the Kaspersky false alarm; look at the blog post below for details) was generating, and found an interesting sentence in there. It says:
This Process sends MIME Email
What? E-Mail? The MobMapUpdater doesn't send any E-Mail! But...it says "MIME Email". And there it became clear why the Prevx heuristic algorithm might believe that the MobMapUpdater does send E-Mail. Of course it doesn't do that, but it does "send" something: it does upload your collected data to the MobMap website, if you wish so. And to do this, it uses an HTTP POST request with MIME encoding for the actual data - and the code component which does that seemed to be enough to trigger the heuristic algorithm which believed that the MIME encoding stuff in the MobMapUpdater just has to be used to send E-Mail (deliberately ignoring that you can do many other things with the MIME encoding standard, like for example upload data to a website).
And if this Prevx heuristics crap can make such a false assumption, the heuristic algorithms at Kaspersky which they probably use to automatically classify new samples (which they get - for example - from the VirusTotal website!) and automatically generate signatures for these samples if the heuristic does believe to have found something suspicious might make the same mistake!
Most people don't really think about how those hundreds of thousands of records in the signature files of all those antivirus scanners are actually created (Kaspersky for example says that they currently have 839572 single records in their database), and if someone actually does think about it, he usually believes that there are many professional virus analysts somewhere in a lab analyzing every single suspected file by hand and crafting signatures to detect them. That may have been true 10 years ago, but today most of the "viruses" and "trojans" in the antivirus databases have never been analyzed by any human being.
Instead, heuristic algorithms try to determine what a suspected file might be doing on a system, and depending on that analysis a file is being classified as containing a virus or not. If such a heuristic finds a new virus which there exists no signature for, it pulls up a new name (hence the naming similarities in the "trojans" mentioned earlier!) and automatically creates a signature for it. No human being involved here - at least not in the everyday process of analyzing new files.
That is how the antivirus companies are able to keep up with the thousands of malware variants that keep popping up every day without having to pay thousands of professional software analysts 24/7. And I think that is also the reason why the MobMapUpdater keeps raising false alarms at the Kaspersky scanner. The last two times, a sample was sent to Kaspersky for manual analysis, and the manual analysis by a skilled virus analyst always quickly led to the conclusion that the file is indeed not a trojan and doesn't contain a trojan, which was followed by Kaspersky removing the signature from their definition files. But the next MobMapUpdater version just generated a new signature through the automated process, and the whole crap started all over again...
So what could I do to prevent all this crap from happening? Make it clear to the heuristics that the MobMapUpdater does not "send MIME Email"! So I ripped out the MIME encoding component I used previously (which actually wasn't written by myself, but a component from a library offered by Borland) and replaced it by a self-written minimalistic MIME encoder that just does what I need to upload the collected data to the website. To be on the safe side I also replaced the Base64 encoder I used previously (also a stock component) with a self-written Base64 encoder. Then I ran the file through VirusTotal (and therefore the Prevx heuristic scanner): 0 alarms, the file is found to be perfectly clean. I hope that these changes suffice to keep the newest version of the MobMapUpdater (v1.77, online for download now!) from appearing in Kaspersky's definition files, so that this odyssey finally has an end!
