I canât believe itâs been over thirteen years since I last wrote about NTFS Alternate Data-Streams.
A lot has changed since then, including the fact that Iâve taken down the site where my download for âsdirâ was listed. But thatâs an old tool, and I donât think we need it any more.
What else has changed is that my wife is studying a number of security courses with the SANS Womenâs Academy, which is an excellent initiative to bring more women into the world of information security, where they, along with the rest of humanity (for whom SANS has other programs), are sorely needed. One of the classes she was studying included a piece on NTFS Alternate Data Streams, or ADS.
An Alternate Data Stream, or ADS, is a parallel stream of data, as the name implies, to the default data stream of a particular file. This default data stream is what most users have spent their lives thinking of as âthe fileâ.
The file is more than just the bytes it contains, in this case. You can go a long way without realising this.
Alternate Data Streams were originally created to support Apple Mac Resource Forks, in files copied from Apple to NTFS and back. Iâm not sure Apple even bothers with them any more, now that theyâve moved to something akin to Linux as their OS.
Created as part of the original NTFS in 1993, these Alternate Data Streams shouldnât be confused with:
Not really easily â at the command prompt, you can use âdir /râ to view the files in your current directory along with all their attendant streams â but you canât combine the â/râ and â/bâ options, so you canât get a really succinct list of all the streams in your system. Hereâs an example listing of a download directory:
In PowerShell, you have more control, and you can even call in to .NET, but you donât need to in order to see file streams. Hereâs a simple command to display just the non-default data streams on files in a particular directory:
Get-ChildItem | Get-Item -Stream * | Where-Object Stream -ne ':$DATA' | Format-Table filename,stream,length
The output this produces looks like this:
Left as an exercise for the reader â how you do this recursively through subdirectories to find all the streams.
The most common ADS on your directory is almost certainly the stream named âZone.Identifierâ, as this is created on every file you download from the web using Internet Explorer, Edge, Chrome, Outlook, or any application that cooperates with Microsoftâs idea of marking files that have been downloaded. If you open Explorer and view properties on a file thatâs been downloaded, youâll see thereâs a checkbox allowing you to âUnblockâ this file, along with a note that it came from another computer. Checking the âUnblockâ box and clicking OK or Apply will remove this Zone.Identifier stream.
This stream is known as the âMark Of The Webâ or âMOTWâ in some documentation, so thatâs another term to use if youâre searching for this stream.
Other stream names I find on my hard drive:
uidStream â I found this on some eBooks in my âMy Kindle Booksâ folder, but whether theyâre specific to the Kindle app, or some other e-reader Iâve used, I canât be certain.
SmartScreen â these are on some downloaded .EXE files, so from the name and file type, Iâll assume this is from virus scanning downloaded EXEs. Since the stream contains just the word âAnaheimâ, Iâm not sure how useful this is.
ms-properties â a binary stream on a few of the JPG images I have on my computer, all of which are photos I took on my Surface Pro.
And some very oddly-named streams on some scanned files, because thereâs just as much of a standard for stream names as there are for file names, and itâs completely a Wild West out there, so the best way to make sure youâre not going to be overwritten by someone elseâs stream is to pick a completely weird and off the wall stream name.
Joking aside, the second of those shows that choosing a GUID is actually a good way to name a stream so it doesnât collide with others â itâs random, and you can make it searchable on the web by documenting it.
Sure enough, if we search for that GUID, thereâs some interesting information to be found at https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/4f3837c4-2f96-40d7-b0bf-80dd1d0b0da0, among other places. This particular GUID is used to include some kind of summary information.
Iâve also read in a couple of places that the Windows File Classification Infrastructure uses ADS to carry information.
It doesnât take much thinking to come up with other uses for alternate data streams. Really any time you might want to associate data with a file, or several files, without bothering the application that might want to read the file. Hereâs some suggestions:
Thinking on this, thereâs a couple of ideas I already have â if I can extract ID3 tags from files and put them into an ADS, itâs going to be quicker and easier to find that information than parsing the entire MP4/MP3/M4A file each time I want to look at the data.
Iâve commented on this before, and Iâve read a lot about how âobviouslyâ viruses will use ADS to hide, or that exfiltration will use ADS to avoid detection, and while thereâs some truth to this idea, I think both threats are overblown.
For exfiltration, the problem is essentially the same as that with using EFS to encrypt a file thatâs being exfiltrated â in order for the data to leave the system, it has to pass through the usual file APIs that your DLP solution is hooked into, and unless your DLP solution is being too smart for its britches, the data will be noticed and blocked. Copying a file from NTFS to FAT or exFAT will destroy the associated ADS data as if it was never there, just as it will destroy EFS encryption.
For virus hiding, while itâs not impossible to execute from an ADS, itâs not particularly easy, and the methods used themselves can trigger your antivirus. To load and execute the data in the ADS, you have to use normal means to load and execute code in a default data stream. And those normal means can be detected by the virus scanner just as easily as they detect any other executable content. If your virus scanner hooks normal load/execute APIs, itâll also intercept the loading and execution of the ADS.
This is probably why thereâs only one virus I found significant information on that uses ADS to hide parts of itself – Backdoor:Win32/Rustock.A â which copies itself into streams off the system32 folder. From the technical description of this virus, itâs also clear that the virus has a fail-back facility for when itâs trying to install itself on a system with no ADS support (really, who installs Windows on a FAT partition? Maybe they mean ReFS, which didnât initially support ADS).
The most likely ADS security threat is still the one for which itâs best known â that of accessing the default data stream of a file by appending â:$DATAâ to the requested filename, and getting around restrictions that an application might have in place.
Years and years ago (1998), this was a trick you could use against IIS to fetch the source code of an ASP page â instead of fetching âpagename.aspâ (which gave you the output of executing the code), youâd fetch âpagename.asp:$DATAâ.
Obviously, IIS fixed this years and years ago, and yet the problem comes up over and over again, in other applications which map incoming requests to files through a simple mapping (file name requested â file name fetched), and which arenât aware of this kind of issue. (On Windows, you can open a file âfor information onlyâ and then query the handle for its canonical name, if you need to write code to get around this â see Writing Secure Code 2nd Edition for details)
So, every now and again, if youâre a hacker and you canât get a file, try getting it with â:$DATAâ at the end of its name.
The command prompt has a few ways to handle Alternate Data Streams with files.
Very limited, as you can tell â you canât do âdir /s/r/bâ to get a list of all the streams, because the /b parameter ignores the /r parameter. You canât directly load an executable from a stream, but you can use another EXE to load it for you (there are examples available online of using WMIC and PSEXEC to do this)
If you absolutely have to remove an alternate data stream from an NTFS file with only Explorer or the Command Prompt, moving it to and from a FAT or exFAT formatted drive (such as a USB stick) will do that, but will also kill any other NTFS properties, such as ownerships, permissions, EFS encryption, etc, as well as killing any audit continuity on the file. I donât recommend this, particularly for those files that you really donât want your name associated with the creation of.
The news is supposedly a little better in PowerShell, which is meant to have built-in support for ADS.
In PowerShell, we use Get-ChildItem to navigate through folders, and Get-Item to look at individual files. Remove-Item is what we use to delete files. Each of these commands has a â-Streamâ parameter, so it seems we are set for our alternate data stream handling.
We can delete a stream from a file as easily(!) as this:
Remove-Item <file> -Stream <stream>
It feels a little weird, but it only deletes the stream, not the file itself.
Seems like this should work to list all streams from our current directory going down, right?
Get-ChildItem -Recurse | Get-Item -Stream * | Where-Object Stream -ne ‘:$DATA’ | Format-Table FileName,Stream,Length
Well, it does most of what weâre looking for.
What it specifically misses is the directories.
Yeah, you can put an alternate data stream on a folder. You canât put a default data stream on a directory, but you can put any number of alternate data streams there.
My PowerShell script wonât find that âwhatnot.txtâ stream. Curiously enough, this is documented at https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-item?view=powershell-5.1 even though itâs clearly an oversight. âThis parameter isnât valid on foldersâ â well, it should be.
Can we use the Remove-Item -Stream parameter to delete streams from directory, even if we canât actually find them using PowerShell?
Sure, but itâs even more scary:
Contrary to what the warning says, the directory and all its children were not deleted, just the stream. Everything is safe and well.
So, what needs fixing?
Oh, yeah, and what on earth is with this lack of support for actual, real, file system features in PowerShell?
And yes, Iâm kind of cheating here, but not much!
Oh, and this folder confuses the command promptâs âdir/r/sâ as well. Note that the directory âAUXâ doesnât have a stream, but when listing the contents of that directory, the directory â.â DOES.
The words, the exploration and examples, the concepts and the thinking, are all shared work between Debbie Lester-Jones and myself.
At some point, when sheâs done with her classes, one of you could be lucky enough to employ her. Or any of the other awesome students of the SANS Womenâs Academy.