I can’t believe it’s been over thirteen years since I last wrote about NTFS Alternate Data-Streams.

A lot has changed since then, including the fact that I’ve taken down the site where my download for “sdir” was listed. But that’s an old tool, and I don’t think we need it any more.

What else has changed is that my wife is studying a number of security courses with the SANS Women’s Academy, which is an excellent initiative to bring more women into the world of information security, where they, along with the rest of humanity (for whom SANS has other programs), are sorely needed. One of the classes she was studying included a piece on NTFS Alternate Data Streams, or ADS.

For the new visitors – what’s an ADS?

An Alternate Data Stream, or ADS, is a parallel stream of data, as the name implies, to the default data stream of a particular file. This default data stream is what most users have spent their lives thinking of as “the file”.

The file is more than just the bytes it contains, in this case. You can go a long way without realising this.

Alternate Data Streams were originally created to support Apple Mac Resource Forks, in files copied from Apple to NTFS and back. I’m not sure Apple even bothers with them any more, now that they’ve moved to something akin to Linux as their OS.

Created as part of the original NTFS in 1993, these Alternate Data Streams shouldn’t be confused with:

• In-stream metadata, such as EXIF data in JPGs, or ID3 tags in MP3s, M4As, MP4s, etc, or Author information in DOCX files – all of these are inside the default data stream
• File metadata, such as owner, security, EFS encryption keys – all of these are in the attributes of the file, in the file table (think of this as being in the directory list, not with the files themselves)

Can I see these streams?

Not really easily – at the command prompt, you can use “dir /r” to view the files in your current directory along with all their attendant streams – but you can’t combine the “/r” and “/b” options, so you can’t get a really succinct list of all the streams in your system. Here’s an example listing of a download directory:

In PowerShell, you have more control, and you can even call in to .NET, but you don’t need to in order to see file streams. Here’s a simple command to display just the non-default data streams on files in a particular directory:

Get-ChildItem | Get-Item -Stream * | Where-Object Stream -ne ':$DATA' | Format-Table filename,stream,length

The output this produces looks like this:

Left as an exercise for the reader – how you do this recursively through subdirectories to find all the streams.

How are ADSs used?

The most common ADS on your directory is almost certainly the stream named “Zone.Identifier”, as this is created on every file you download from the web using Internet Explorer, Edge, Chrome, Outlook, or any application that cooperates with Microsoft’s idea of marking files that have been downloaded. If you open Explorer and view properties on a file that’s been downloaded, you’ll see there’s a checkbox allowing you to “Unblock” this file, along with a note that it came from another computer. Checking the “Unblock” box and clicking OK or Apply will remove this Zone.Identifier stream.

This stream is known as the “Mark Of The Web” or “MOTW” in some documentation, so that’s another term to use if you’re searching for this stream.

Other stream names I find on my hard drive:

uidStream – I found this on some eBooks in my “My Kindle Books” folder, but whether they’re specific to the Kindle app, or some other e-reader I’ve used, I can’t be certain.

SmartScreen – these are on some downloaded .EXE files, so from the name and file type, I’ll assume this is from virus scanning downloaded EXEs. Since the stream contains just the word “Anaheim”, I’m not sure how useful this is.

ms-properties – a binary stream on a few of the JPG images I have on my computer, all of which are photos I took on my Surface Pro.

And some very oddly-named streams on some scanned files, because there’s just as much of a standard for stream names as there are for file names, and it’s completely a Wild West out there, so the best way to make sure you’re not going to be overwritten by someone else’s stream is to pick a completely weird and off the wall stream name.

Joking aside, the second of those shows that choosing a GUID is actually a good way to name a stream so it doesn’t collide with others – it’s random, and you can make it searchable on the web by documenting it.

Sure enough, if we search for that GUID, there’s some interesting information to be found at https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/4f3837c4-2f96-40d7-b0bf-80dd1d0b0da0, among other places. This particular GUID is used to include some kind of summary information.

I’ve also read in a couple of places that the Windows File Classification Infrastructure uses ADS to carry information.

Other possible uses of ADS

It doesn’t take much thinking to come up with other uses for alternate data streams. Really any time you might want to associate data with a file, or several files, without bothering the application that might want to read the file. Here’s some suggestions:

• Metadata for files that don’t support embedding ID3 or similar tags
• Preview clips, sample images
• Maintaining hashes of file contents so as to be aware of file changes (like the “TripWire” application which detects changes to files on servers that shouldn’t change)
• Document tracking information

Thinking on this, there’s a couple of ideas I already have – if I can extract ID3 tags from files and put them into an ADS, it’s going to be quicker and easier to find that information than parsing the entire MP4/MP3/M4A file each time I want to look at the data.

Is ADS a security threat?

I’ve commented on this before, and I’ve read a lot about how “obviously” viruses will use ADS to hide, or that exfiltration will use ADS to avoid detection, and while there’s some truth to this idea, I think both threats are overblown.

For exfiltration, the problem is essentially the same as that with using EFS to encrypt a file that’s being exfiltrated – in order for the data to leave the system, it has to pass through the usual file APIs that your DLP solution is hooked into, and unless your DLP solution is being too smart for its britches, the data will be noticed and blocked. Copying a file from NTFS to FAT or exFAT will destroy the associated ADS data as if it was never there, just as it will destroy EFS encryption.

For virus hiding, while it’s not impossible to execute from an ADS, it’s not particularly easy, and the methods used themselves can trigger your antivirus. To load and execute the data in the ADS, you have to use normal means to load and execute code in a default data stream. And those normal means can be detected by the virus scanner just as easily as they detect any other executable content. If your virus scanner hooks normal load/execute APIs, it’ll also intercept the loading and execution of the ADS.

This is probably why there’s only one virus I found significant information on that uses ADS to hide parts of itself – Backdoor:Win32/Rustock.A – which copies itself into streams off the system32 folder. From the technical description of this virus, it’s also clear that the virus has a fail-back facility for when it’s trying to install itself on a system with no ADS support (really, who installs Windows on a FAT partition? Maybe they mean ReFS, which didn’t initially support ADS).

Your most likely ADS security threat

The most likely ADS security threat is still the one for which it’s best known – that of accessing the default data stream of a file by appending “:$DATA” to the requested filename, and getting around restrictions that an application might have in place.

Years and years ago (1998), this was a trick you could use against IIS to fetch the source code of an ASP page – instead of fetching “pagename.asp” (which gave you the output of executing the code), you’d fetch “pagename.asp:$DATA”.

Obviously, IIS fixed this years and years ago, and yet the problem comes up over and over again, in other applications which map incoming requests to files through a simple mapping (file name requested ≊ file name fetched), and which aren’t aware of this kind of issue. (On Windows, you can open a file “for information only” and then query the handle for its canonical name, if you need to write code to get around this – see Writing Secure Code 2nd Edition for details)

So, every now and again, if you’re a hacker and you can’t get a file, try getting it with “:$DATA” at the end of its name.

Command line ADS handling

Command Prompt

The command prompt has a few ways to handle Alternate Data Streams with files.

1. Pass the stream name to an executable – maybe it’ll handle it.
   Notepad is a good example of this – you can directly edit streams by running “notepad file:stream”.
   Exercise for the reader – create a file called “C”. Open a stream called “foo.txt” on file “C” using notepad.
2. Redirection – you can always use redirection to and from streams
   ”dir > foo.txt:bar.dat” will create a stream on file foo.txt
   ”more < foo.txt:bar.dat” will display the stream back to you.
3. Some built-in commands may have stream support built in – but this is very limited.
   As an example, you’ve seen the “dir /r” command I used above, which lists streams.

Very limited, as you can tell – you can’t do “dir /s/r/b” to get a list of all the streams, because the /b parameter ignores the /r parameter. You can’t directly load an executable from a stream, but you can use another EXE to load it for you (there are examples available online of using WMIC and PSEXEC to do this)

If you absolutely have to remove an alternate data stream from an NTFS file with only Explorer or the Command Prompt, moving it to and from a FAT or exFAT formatted drive (such as a USB stick) will do that, but will also kill any other NTFS properties, such as ownerships, permissions, EFS encryption, etc, as well as killing any audit continuity on the file. I don’t recommend this, particularly for those files that you really don’t want your name associated with the creation of.

PowerShell

The news is supposedly a little better in PowerShell, which is meant to have built-in support for ADS.

In PowerShell, we use Get-ChildItem to navigate through folders, and Get-Item to look at individual files. Remove-Item is what we use to delete files. Each of these commands has a “-Stream” parameter, so it seems we are set for our alternate data stream handling.

We can delete a stream from a file as easily(!) as this:

Remove-Item <file> -Stream <stream>

It feels a little weird, but it only deletes the stream, not the file itself.

Seems like this should work to list all streams from our current directory going down, right?

Get-ChildItem -Recurse | Get-Item -Stream * | Where-Object Stream -ne ‘:$DATA’ | Format-Table FileName,Stream,Length

Well, it does most of what we’re looking for.

What it specifically misses is the directories.

Wait, what, directories? Folders?

Yeah, you can put an alternate data stream on a folder. You can’t put a default data stream on a directory, but you can put any number of alternate data streams there.

My PowerShell script won’t find that ‘whatnot.txt’ stream. Curiously enough, this is documented at https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-item?view=powershell-5.1 even though it’s clearly an oversight. “This parameter isn’t valid on folders” – well, it should be.

Can we use the Remove-Item -Stream parameter to delete streams from directory, even if we can’t actually find them using PowerShell?

Sure, but it’s even more scary:

Contrary to what the warning says, the directory and all its children were not deleted, just the stream. Everything is safe and well.

Summarising the bugs

So, what needs fixing?

1. The command prompt needs a DEL command that deletes a stream. Because, really, functionality.
2. While the command prompt can do a DIR/R, it needs to work with the /B parameter, so that DIR/S/R/B|FINDSTR “:” can be used to find all the streams.
3. Windows Explorer should have a way to find streams,  browse them, open them in an application, and delete them.
4. PowerShell’s Get-Item needs to support the -Stream parameter on folders, because NTFS allows streams on folders.
5. PowerShell’s Remove-Item needs to just delete a stream when you specify -Stream and not scare the living daylights out of you.

One final bug

Oh, yeah, and what on earth is with this lack of support for actual, real, file system features in PowerShell?

And yes, I’m kind of cheating here, but not much!

Oh, and this folder confuses the command prompt’s “dir/r/s” as well. Note that the directory “AUX” doesn’t have a stream, but when listing the contents of that directory, the directory “.” DOES.

Attribution

The words, the exploration and examples, the concepts and the thinking, are all shared work between Debbie Lester-Jones and myself.

At some point, when she’s done with her classes, one of you could be lucky enough to employ her. Or any of the other awesome students of the SANS Women’s Academy.