A security problem in Windows has been reported which affects any EXEs downloaded from the internet and run from the download folder. Setup packages probably are especially in danger.
How the vulnerability works
When you start an EXE which loads additional DLLs, Windows starts looking for these DLLs in the same directory as the EXE first, and only if the DLL isn’t found there it continues searching in the System folder and other folders in the PATH environment.
When an EXE is downloaded with a web browser, it gets stored in the standard Downloads folder. If an attacker managed to place a DLL in that Downloads folder (e.g. using a drive by download) and the name of the DLL happens to match a DLL that the EXE loads, the malicious DLL is used instead of the system DLL. The malicious DLL may have been placed in the Downloads folder long before, sitting there waiting to be loaded by some downloaded EXE.
Since the attack injects the malicious code as a DLL, it doesn’t change the EXE’s binary. So the digital signature stays intact and everything looks legit to the user.
Setups could be preferred targets
In general, any EXE which is downloaded and executed could be attacked in this way, including self extracting zip files and the like. But the attacker needs to know the name of a DLL that the EXE wants to load. Setups often load a set of standard DLLs, which makes predicting the DLL name easier. And setups typically request administrative privileges, giving the malicious DLL even more power.
Pure MSI files are not affected
The vulnerability only affacts EXEs. If your setup comes as a MSI file without setup.exe, you should be safe.
Workarounds and Fixes
Microsoft Security Bulletin MS16-007 includes some security updates related to “DLL Loading”. It’s unclear to me whether these address this vulnerability.
[UPDATE Jan 22nd: According to Rob Mensching of FireGiant, MS16-007 does not address this vulnerability.]
FireGiant, the company supporting the WiX Toolset, reported this vulnerability to Microsoft. In their blog they posted technical details about the vulnerability and possible mitigations. An update for WiX 3.10 has also been released. Other manufacturers of setup auhtoring tools will probably follow.
[UPDATE Jan 22nd: Advanced Installer posted about this problem already on January 7th (sorry I forgot to mention that). Like FireGiant, they recommend naming your installer something other than setup.exe. They also announced that the next version will include mitigations to reduce the risk of this vulnerability.]
[UPDATE Jan 25th: Flexera Software has published a knowledge base article about Best Practices to Avoid Windows Setup Launcher Executable Issues and released a hotfix for InstallShield and AdminStudio.]