Surprisingly the answer may well be yes. And it’s not just different names but potentially different names in different directories that can actually refer to the same file.
Unix and related operating systems have have been able to do this for decades. What most people don’t realize is that Windows NT and its successors also support this; or rather the NTFS filesystem supports this. “This” is the concept of “hard links”. To understand what hard links are, we need to take a look at what it means to be a directory entry for a file on disk.
Become a Patron of Ask Leo! and go ad-free!
When you open up Windows Explorer to view the contents of a folder or do a “DIR” in a Command shell, you’re looking a a list of the files in that folder or a “directory” of those files. The contents of each file live somewhere else on the hard disk; but the file’s name, date, and other information is stored in its directory entry.
Perhaps the most important bit of information stored in the directory listing for a file the is location of its contents on the hard disk. The operating system uses the directory information to actually retrieve the contents when you read or modify it.
A common term is that the directory entry “points to” or “links to” the file as illustrated by the diagram below:
So if each file’s contents is out on the hard disk and a directory entry merely points to it then what prevents a second directory entry from pointing to the same file contents?
In a word: nothing. At least not under NTFS. You can have many directory entries pointing to the same file contents much like this diagram:
If you change the contents of the file using any of the directory entries that point to it then all of the directory entries will reflect the change.
Creating additional directory entries to point to a file is easy under Windows though the command line is a bit long. You start by creating the file however you normally would – that will create the single directory entry and the file contents as illustrated in the first diagram above. To create an additional directory entry that links to the same file in a Command Shell enter:
fsutil hardlink create newname existingfilename
“existingfilename” is the name of the file that already exists on disk. “newname” is the filename to be used in the additional directory entry to be created. So for example, if I have “c:\my documents\latest paper.doc” and I want that same file to also appear elsewhere then I might enter:
fsutil hardlink create “c:\shared files\paper.doc” “c:\my documents\latest paper.doc”
“c:\my documents\latest paper.doc” and “c:\shared files\paper.doc” are now two names for the same file. Change one and it shows up in the other.
Delete one and the other remains. Deleting a directory entry only removes the directory entry. It’s only when the last directory entry referencing a file is removed that the file itself is finally removed from the disk.
One last limitation: hard links only work on the same drive. So for example you can’t have linked directory entries on D: for files on C:.
is there a program that will search for these doubles
The short answer is no … at least as far as I know. I did actually look for such a program a couple of years ago, as we were using hard links heavily for one of our applications within Microsoft. The problem is that there is no central database, so to speak, and no backwards pointer from the file contents to it’s directory entries. The only way to locate all directory entries that point to the same file is to write a program that walks the entire hard drive, looking at *ALL* directory entries to see if they happen to point to the same file.
Leo
Is this possible on Win 2000 ? I tried but fsutil seems to be not there.
Yes and no. The operating system and NTFS supports it. However there are no command-line tools that come with Win2k.
There is a utility in 2003 Resouce Kit: Hlscan.exe
hmm.. I got a problem similar to what’s mentioned above. I was deleting duplicate files from my Ipod and I came across a music file that exists in folder “.” which is a subfolder of “Enigma” This folder (directory) doesn’t seem to exist when I try to navigate to it via windows explorer, but through itunes I can reach the file AND it shows up as a legitimate sub-directory and file. I thought it was some weird windows glitch that dupicated the directory, and I tried to delete it. Thats when I discovered that this sub-directory and it’s files actually point back to the parent directory. Deleting sub-directory “.” deletes the original files. I’m perplexed on how to solve this so that directory “.” no longer exists. I’ve read the info on HLscan.exe seems only to list the files. Anyone know how to delete the “other” files names
Is there any way to do this for directories, instead of files? E.g. I want “C:\My Documents” to be a hard link to “C:\Documents and Settings\Kevin J. Wangler\My Documents”. I tried using the same command and it just said ‘Error: Access is denied’. So, I assume this isn’t possible, but I thought I’d ask. Thanks!
To answer my own question: Yes, it can be done! I just found a freeware tool called Winbolic Link which allows you to create symlinks (symbolic links), much like in *NIX. It’s available at http://www.pearlmagik.com/winbolic/ (I’m not affiliated with PearlMagik or Winbolic Link; it’s just something I found doing a Google search. I just tried it and created a C:\My Documents symlink, and it worked great.)
Is there a SAFE (or ‘least dangerous’) way to DEFRAGMENT a hard drive on which I have created a great deal of NTFS hard links? I understand that Windows Explorer does not ‘know’ whether it is looking at a ‘hard link’ of a file or not, and hence I am worried that the intricate system of hard links that I have built up within my hard drive will be damaged or lost completely when attempting to defragment the drive.
Alternatively, is there a way to ‘back up’ the NTFS master file table in such a way that if the hard link structure is damaged or destroyed by defragmenting the drive, that the back-up version of that original (pre-defragmentation) file table can be ‘restored’ somehow ?
I wish I could afford to simply back up the entire drive contents prior to defragmenting it (just in case of such an unwanted degradation of the hard-link structure), but unfortunately I have approximately a terabyte of information stored on my internal hard drives, and I literally cannot afford another terabyte of storage at this time !
Defragging shouldn’t affect hardlinks.
I am trying to create a batch file that will rename all dvr-ms files to mpg in a specific directory on a daily basis, instead of using the copy command and doubling the used disk space per file (about 4gb), I am trying to use the –fsutil hardlink create– command to create a duplicate filename just with a different extension in the same directory. HOWEVER… when I use the “*.” identifier to perform this action, it tells me that I cant do that. This is the string that I am entering: “fsutil hardlink create d:\record~1\*.mpg(new) d:\record~1\*.dvr-ms(old)” what am I doing wrong?
You’ll need to look into the FOR statement and execute the command one file at a time.
A simple example of FOR:
FOR %F IN (*.MPG) DO echo %F
I think you should be able to construct a FOR that does what you want. It’ll take some experimentation, so do back up your files :-).
Cygwin (www.cygwin.com) can be used to access the linux file handling commands making hard links usage much simpler. It also works on windows 2000.
For example, rsync can be used to create what appear to be multiple full backups. Since it can use hard links, the backups are actually incremental “under the hood”, thus potentially saving a massive amount of backup space.
For example:
rsync -a –delete –link-dest=prevBackupDir dirToBackup currBackupDir
will backup dirToBackup into currBackupDir. Any files that have not changed since prevBackupDir was made will be created as hard links to the prevBackupDir version. I.e. (almost) no disk space is required for these unchanged files.
I use this to keep 10 full nightly backups of a 23G directory on an external drive. It only takes rougly 31G of drive space instead of 230G.
I do find this windows scheme of referring to a particular file from various places in the directory/folder system to be very confusing. One place where it shows up is when doing a Search for a particular file. One will find anything up to half a dozen references to it – occasionally more!
I like to put My Documents on a different drive from the operating system and this confuses Windows and, as a result, every so often, I have to go to the various other ‘stores’ of My Documents and collect/transfer them to where I want them to be! Windows does have a will of its own and it takes a lot of effort to beat it and get it to do what you want, particularly in the case the storage of documents and similar files.
I say, why use hardlinks (or symlinks) at all, when you can use Windows shortcuts?
Because those only work in windows UI and programs that know how to decode shortcuts. Hardlinks work in command prompt, and everywhere else, as they are in use indistinguishable from the original.