Monday, August 8, 2011

Authenticode and Antivirus Detection part 2

Update Nov 2013: Another follow up post: Authenticode and Antivirus Detection Revisited

After Shane's comments on the Authenticode and Antivirus Detection post I thought I'd run some more tests. I wanted to try and figure out how much of the observed detection difference were because some extra bytes had been added and how much was due to special handling of signed binaries.

I found an archive of malware online and created four sets of samples. Set one was the malware without any changes, set two was after the binaries had been signed with the TEST1 certificate, set three was signed with a TEST2 certificate that was similar to TEST1 but was only valid from 1975 - 2009 and set four had a random blob of 32 bytes appened to the end. Using the VirusTotal API and Bryce Boe's python script I ran each of the sets against the VirusTotal antivirus suite.

The resulting statistics are here, showing the number of AV positives, the format is:
 "HASH [SET1, SET2, SET3, SET4] [SET1 - SET2, SET1 - SET4]"

And here are the first 10 entries (ordered by decreasing "SET1 - SET2" value):

DB1D5E...34573 ['28', '10', '16', '22'] ['18', '6']
FDFB86...1FE0C ['34', '18', '18', '27'] ['16', '7']
6D48A7...F4880 ['36', '20', '22', '33'] ['16', '3']
CA9C3E...ED072 ['31', '16', '16', '23'] ['15', '8']
8798FA...8755B ['35', '20', '20', '32'] ['15', '3']
1011ED...0DB18 ['35', '20', '20', '33'] ['15', '2']
DA01D0...C899D ['31', '17', '16', '28'] ['14', '3']
CC3B7D...228D1 ['37', '23', '23', '34'] ['14', '3']
CADD90...CE9C4 ['35', '21', '21', '31'] ['14', '4']
B6BBE8...8CD10 ['32', '18', '18', '29'] ['14', '3']

General observations:
  • Adding either an authenticode signature or random data would defeat several engines
  • Very rarely would the signing certificate's validity influence the score
  • For some reason adding the random data occasionally resulted in more signatures being hit and considering that the same data was added to each sample I'm not sure what happened there.
  • This test primarily tests the AV signature engines and not their runtime or heuristic scanners
  • The VirusTotal API limit of 20 requests each 5 minutes sounds like a lot until you run tests like this.
Really what I've learnt from this is that AV signatures are even more fragile than I realised. To get a proper look at how AV treats authenticode signed binaries I think I'd need to evaluate all of each AV's modules and not just the signature engine.

Saturday, August 6, 2011

Tavis Ormandy's Sophail paper

On the topic of antivirus, Tavis Ormandy has recently released a paper looking into the internals of Sophos. It's quite scathing and very interesting, check it out: Sophail: A Critical Analysis of Sophos Antivirus

Authenticode and Antivirus Detection

Update Nov 2013: Follow up posts: Authenticode and Antivirus part 2, Authenticode and Antivirus Detection Revisited

It turns out that many antivirus engines white list authenticode signed binaries regardless of the trustworthiness of the signature. Here's an experiment that I performed, feel free to play along at home (remember to be careful when working with malware).

Step 1: Find some malware
This was actually the most time consuming step, a lot of places talk about malware and offer large archives of malware samples to download. Even so, it took me a good 15 minutes to find a malicious windows executable that I could download from a site without a password, registration or other nonsense. In the end I found a site that lists live drive by download sites and I grabbed an EXE before the particular malware host went down. Sadly I can't find the link to the index site I was using, I'm sure a little bit of Googling will allow you to retrace my steps.

I ended up with freedom.exe md5sum: ba87b562c829b7095bfb9e0db7a39890

Step 2: Confirm that it is detected by Antivirus
For this to work you need to know that your malware sample is detected by antivirus engines so I recommend submitting it to VirusTotal or similar service. Alternative if you have the resources run it against your local battery of antivirus installs.

Freedom.exe was detected under a variety of names, Microsoft Security Essentials calls it Trojan:Win32/Danginex. The results were 36/43 (83.7%) considered Freedom.exe malware.

Step 3: Generate a code signing certificate
I don't have a proper code signing certificate handy so I thought I'd generate a self-signed certificate for the test. I used makecert.exe and pvk2pfx.exe from the Windows SDK 7.1 and the following commands:

makecert -r -pe -$ individual -n CN=TEST1 -sv test1.pvk test1.cer
pvk2pfx -pvk test1.pvk -spc test1.cer -pfx test1.pfx

Step 4: Sign the malware sample
Copy the sample to a new filename and then use signtool.exe to add the authenticode signature saying that TEST1 is responsible for this file.

signtool sign /f test1.pfx freedo-signed-test1.exe

Step 5: See what AV thinks of this new file
Submit your new file to VirusTotal and see what happens. In the case of Freedom.exe the detection rate fell from 83.7% to 27.9% (12/43). Most of the big names in the AV community (with a couple of notable exceptions) were quite happy to ignore Freedom.exe once it had been signed.

The Antivirus engines that changed their minds about freedom.exe are:
AhnLab-V3, AVG, BitDefender, CAT-QuickHeal, Comodo, Emsisoft,
F-Secure, Fortinet, Ikarus, K7AntiVirus, McAfee,
McAfree-GW-Edition, Microsoft, Norman, nProtect, PCTools, Rising,
Sophos, Symantec, TheHacker, TrendMicro, TrendMicro-HouseCall,
VIPRE, ViRobot

Notably Kaspersky flagged both the original and modified samples as Trojan-Clicker.Win32.Agent.shx and ClamAV among 7 others did not flag either sample.

Conclusion: What have we learnt?
Signed executables are more likely to be considered benign by antivirus engines. Signed executables are probably excluded by policy for performance reasons but it is possible (but unlikely) that instead that the addition of the Authenticode block at the end of the file is disrupting the signatures used by the engines. I hope that in the future that if vendors are going to exclude signed binaries that they at least check to see if the certificate used to sign the binary is trusted.