Authenticode and Antivitus Detection

Editors note: This used to be multiple blogposts that have been collected together in chronological order

It turns out that many antivirus engines white list authenticode signed binaries regardless of the trustworthiness of the signature. Here's an experiment that I performed, feel free to play along at home (remember to be careful when working with malware).

Step 1: Find some malware
This was actually the most time consuming step, a lot of places talk about malware and offer large archives of malware samples to download. Even so, it took me a good 15 minutes to find a malicious windows executable that I could download from a site without a password, registration or other nonsense. In the end I found a site that lists live drive by download sites and I grabbed an EXE before the particular malware host went down. Sadly I can't find the link to the index site I was using, I'm sure a little bit of Googling will allow you to retrace my steps.

I ended up with freedom.exe md5sum: ba87b562c829b7095bfb9e0db7a39890

Step 2: Confirm that it is detected by Antivirus
For this to work you need to know that your malware sample is detected by antivirus engines so I recommend submitting it to VirusTotal or similar service. Alternative if you have the resources run it against your local battery of antivirus installs.

Freedom.exe was detected under a variety of names, Microsoft Security Essentials calls it Trojan:Win32/Danginex. The results were 36/43 (83.7%) considered Freedom.exe malware.


Step 3: Generate a code signing certificate
I don't have a proper code signing certificate handy so I thought I'd generate a self-signed certificate for the test. I used makecert.exe and pvk2pfx.exe from the Windows SDK 7.1 and the following commands:


makecert -r -pe -$ individual -n CN=TEST1 -sv test1.pvk test1.cer
pvk2pfx -pvk test1.pvk -spc test1.cer -pfx test1.pfx


Step 4: Sign the malware sample
Copy the sample to a new filename and then use signtool.exe to add the authenticode signature saying that TEST1 is responsible for this file.


signtool sign /f test1.pfx freedo-signed-test1.exe



Step 5: See what AV thinks of this new file
Submit your new file to VirusTotal and see what happens. In the case of Freedom.exe the detection rate fell from 83.7% to 27.9% (12/43). Most of the big names in the AV community (with a couple of notable exceptions) were quite happy to ignore Freedom.exe once it had been signed.


The Antivirus engines that changed their minds about freedom.exe are:
AhnLab-V3, AVG, BitDefender, CAT-QuickHeal, Comodo, Emsisoft,
F-Secure, Fortinet, Ikarus, K7AntiVirus, McAfee,
McAfree-GW-Edition, Microsoft, Norman, nProtect, PCTools, Rising,
Sophos, Symantec, TheHacker, TrendMicro, TrendMicro-HouseCall,
VIPRE, ViRobot

Notably Kaspersky flagged both the original and modified samples as Trojan-Clicker.Win32.Agent.shx and ClamAV among 7 others did not flag either sample.

Conclusion: What have we learnt?
Signed executables are more likely to be considered benign by antivirus engines. Signed executables are probably excluded by policy for performance reasons but it is possible (but unlikely) that instead that the addition of the Authenticode block at the end of the file is disrupting the signatures used by the engines. I hope that in the future that if vendors are going to exclude signed binaries that they at least check to see if the certificate used to sign the binary is trusted.


 

After Shane's comments on the Authenticode and Antivirus Detection post I thought I'd run some more tests. I wanted to try and figure out how much of the observed detection difference were because some extra bytes had been added and how much was due to special handling of signed binaries.

I found an archive of malware online and created four sets of samples. Set one was the malware without any changes, set two was after the binaries had been signed with the TEST1 certificate, set three was signed with a TEST2 certificate that was similar to TEST1 but was only valid from 1975 - 2009 and set four had a random blob of 32 bytes appened to the end. Using the VirusTotal API and Bryce Boe's python script I ran each of the sets against the VirusTotal antivirus suite.

The resulting statistics are here, showing the number of AV positives, the format is:
 "HASH [SET1, SET2, SET3, SET4] [SET1 - SET2, SET1 - SET4]"

And here are the first 10 entries (ordered by decreasing "SET1 - SET2" value):

DB1D5E...34573 ['28', '10', '16', '22'] ['18', '6']
FDFB86...1FE0C ['34', '18', '18', '27'] ['16', '7']
6D48A7...F4880 ['36', '20', '22', '33'] ['16', '3']
CA9C3E...ED072 ['31', '16', '16', '23'] ['15', '8']
8798FA...8755B ['35', '20', '20', '32'] ['15', '3']
1011ED...0DB18 ['35', '20', '20', '33'] ['15', '2']
DA01D0...C899D ['31', '17', '16', '28'] ['14', '3']
CC3B7D...228D1 ['37', '23', '23', '34'] ['14', '3']
CADD90...CE9C4 ['35', '21', '21', '31'] ['14', '4']
B6BBE8...8CD10 ['32', '18', '18', '29'] ['14', '3']


General observations:

  • Adding either an authenticode signature or random data would defeat several engines
  • Very rarely would the signing certificate's validity influence the score
  • For some reason adding the random data occasionally resulted in more signatures being hit and considering that the same data was added to each sample I'm not sure what happened there.
  • This test primarily tests the AV signature engines and not their runtime or heuristic scanners
  • The VirusTotal API limit of 20 requests each 5 minutes sounds like a lot until you run tests like this.
Really what I've learnt from this is that AV signatures are even more fragile than I realised. To get a proper look at how AV treats authenticode signed binaries I think I'd need to evaluate all of each AV's modules and not just the signature engine.

It's time to revisit code signing and antivirus detection! Two years ago I looked into whether or not Authenticode signatures (Microsoft object code signing for PE files) influenced the decisions of antivirus engines.

In the first part I described the process of finding, signing and testing some malware with VirusTotal and it appeared that adding an Authenticode signature to a known piece of malware drastically lowered its detection rate. After an astute observation in the comments and some thinking about it, I decided this was more likely due to the fragility of the signatures created by antivirus vendors. In the second part I test this theory by using the VirusTotal API to test 100 malware samples to compare the response of adding an Authenticode signature to changing the binary in other ways. The result largely confirmed that rather than code signing that was defeating the antivirus scans, it was that the binary was changing at all.

These results were ultimately unsatisfying, partly because of the surprising fragility of the antivirus signatures but also because of the test methodology. VirusTotal is a wonderful resource but is only testing the core antivirus engine of these security products which these days is only a small part of the protective services provided by security software. Additionally, the certificate that the malware was signed with was not endorsed by any certificate authority and was not deemed trusted by the operating system.

This time around I investigated how the actual security products responded, and used a trusted certificated to sign the malware. Unlike last time, the number of engines tested is much lower and the number of samples used is only one. Turns out that this kind of testing is very time intensive if you don't already have the infrastructure set up. While I did have some free time during the US Government shutdown while waiting for a US visa, there's a limit to how many virtual machines I felt like setting up.

Part 1 - The Malware
This time the malware sample used is VPN-Pro.exe (MD5: 8eda7dfa4ec4ac975bb12d2a3186bbeb) as contributed by the redoubtable @headhntr to the Syrian Malware Samples Project. As described by the Citizenlab analysis it is a trojanized version of Freegate 7.35 written using .NET 3.5 that drops the ShadowTech RAT. The campaign was targeted at dissidents in Syria, make sure that you check out the analysis, it's fascinating but a little outside the scope of this post.

When Citizenlab submitted VPN-Pro.exe to VirusTotal in June 2013 it was detected by 5/46 antivirus engines. When I checked in October the VirusTotal report had been updated to show detection by 34/47 antivirus engines.

Part 2 - The Antivirus Suites
The test environment is a series of Windows 8.1 Virtual Machines with the following security software suites installed.

  • Sophos Endpoint Client Protection
  • McAfee All Access
  • Norton 360
  • Windows Defender (as installed by default with Windows 8.1)
In addition Chrome was installed in each VM, and each sample was submitted to VirusTotal.
 
Part 3 - The Transformations
Six versions of the sample were tested, the original and five transformed versions.
 
1. Padded
VPN-Pro.exe with 1024 'A' characters appended to the end.
SHA256: 54d9f5767ec3a7aba6754dacc998d57bb54e793750f2e5b1e63e37cc9c43da6e
 
2. Random Padding
VPN-Pro.exe with random bytes added to match the length of the signed version.
SHA256: 8428aa9dfa69438d98b0008b0dc7c9e8135889d893a77d5536aacf8b7e1ad6e7
 
3. Authenticode (Self-signed certificate)
VPN-Pro.exe signed with a test certificate (as describe in part 1 of this series)
SHA256: f8efd5ad3ad13e218b71dece73766c87e57c80d5f7a1fd78f319312baa11bcf7
 
4. Damaged Authenticode (Self-signed certificate)
VPN-Pro.exe as prepared in number 3 but with roughly 10% of the signature bytes replaced randomly.
SHA256: 1ddf2de1bb8d289b6b77843bc2b9a685d31d0f17371691e3f4b81faa383c7769  
 
5. Authenticode (Trusted certificate)
VPN-Pro.exe signed with a trusted code signing certificate from StartSSL.com (thanks to the anonymous benefactor that helped me with this part).
SHA256: f8efd5ad3ad13e218b71dece73766c87e57c80d5f7a1fd78f319312baa11bcf7
 
Part 4 - The Method
Each VM had the trial version (as found on the vendor's website) of the security software installed. Windows and the security software were then updated. Next Chrome was used to download each sample and the reactions of the browser and security software were recorded. If there was no reaction from the security software or browser a manual scan was initiated where possible. In the case of Windows Defender the same tests were undertaken using Internet Explorer as well as Chrome.
 
Part 5 - The Results

Summary
 OriginalPaddedRandom PaddingDamaged SignatureSelf SignedTrusted Signature
McAfee All Access      
Norton 360      
Sophos Endpoint      
Windows Defender     Mild Warn
Virus Total34/477/477/467/477/478/47
       
 Nothing detected    
 Warning, this may harm your PC   
 Reputation based detection   
 Explicitly marked as virus   

McAfee All Access
 
The original sample was detected as malware and automatically removed.
 
All other samples were detected as malware and quarantined.
 
Considering that on the VirusTotal scan McAfee did not detect any of the transformed samples, my guess is that the Quarantined dialog is shown on heuristic or binary reputation based matches while the automated removal dialog is for signature based matches.
 
Norton 360
 
The original sample was detected as malware and automatically removed.
 
The transformed samples were also all detected and automatically removed.
The difference being that all the transformed files were detected as the threat "WS.Reputation.1" and the Threat Type of "Insight Network Threat". This suggests to me that the cloud binary reputation service is flagging these files as harmful largely because they have note been seen before. Again, the Symantec engine did not detect the transformed files during the VirusTotal submission (I assume that Norton 360 uses the Symantec antivirus engine).
 
Sophos Client Endpoint Protection
 
Again, the original sample was detected and removed.
 
In fact all the transformations (except one) of VPN-Pro.exe were detected in the same way as the original and were tagged as Mal/Generic-S. The original was flagged as Mal/Generic-S on VirusTotal as well but the transformations weren't likewise flagged at the time, unsure whether this is due to some fuzzy matching or updated signatures.
 
However, the Authenticode version with the trusted certificate was downloaded without complaint. Considering that the self signed version was flagged as malicious, I'm drawn to conclude that the validity of the signature was taken into account.
 

 

A manual scan was run after the initial download completed.
 
Windows Defender
 
Windows Defender on Internet Explorer gave the largest variety of messages, here's all six:
 
 
Like all the other security suites, the original was flagged as a virus and removed. What happens with the transformed versions is rather more interesting. Firstly the padded version was still detected as a virus, while the randomly padded one wasn't (the random padding was longer than the non-random padding). Unique to IE the damaged signature transformation was reported as a 'corrupt or invalid' signature and treated differently to the random padding transformation. The self-signed Authenticode transformation was flagged as "not commonly downloaded" rather than a virus, and the the trusted certificate Authenticode transformation was flagged the same way but with a yellow bar rather than a red one.

The diversity of messages here was surprising, clearly the signature of the file is being examined and being combined with some cloud based binary reputation system (Smartscreen filter?) before a determination is given to the user. It's worth noting that a non-malicious, unsigned, uncommon binary gave the same message as the signed (untrusted) executable and that a non-malicious, unsigned, common binary (putty.exe) gave no warning message. This means that the malicious, signed binary landed somewhere in between these two cases.

Part 6 - Conclusion
First a caveat: with only a single malware sample and a small handful of security suites we can not come to any sweeping conclusions.
 
However, it looks like Windows Defender / Internet Explorer as well as Sophos take into account Authenticode signatures when scanning executables. All tested security suites seem to have very fragile signature driven engines that were defeated by almost any change to the sample but these systems are backed up by heuristic systems that are at least partially powered by a cloud based binary reputation mechanism. Windows Defender and Sophos both differentiated between untrusted Authenticode signatures and trusted signatures and Windows Defender differentiated between Authenticode signatures, a corrupted Authenticode signature, and arbitrary appended data.
 

This article was updated on December 25, 2024