Why is the Same File a Different Size in Different Places?

This is something that’s confused computer users for many years: the exact same file can show as taking up a different amount of space, depending on where you look and the characteristics of different disk drives.

Copy that file online and things get even more confusing.

I don’t think I’ve ever seen this be something to worry about. Regardless of the differences, your file is still your file, regardless of where it’s stored.

• Disk space is allocated one “cluster” at a time. Even a one-byte file takes up at least one cluster of space.
• Cluster size is configurable when a disk is formatted, and generally ranges from 512 to 131,072 bytes.
• Different utilities show disk space differently.
• Online services hide all that and simply show your file sizes.

File size versus file size

I’ll use a one-byte file as my example: one-byte-file.txt.

I used Command Prompt specifically because it shows the file size as being exactly one byte.

Unlike Windows File Explorer:

Here you can see the file listed as “1KB” (1024 bytes) in size.

So, what it is it? One byte or over a thousand?

Well, in a way, it’s both.

To understand why, we need to look at how disk space is allocated.

Clusters

Data on hard disks is stored in sectors of 512 or 4,0961 bytes at a time. This physical organization maximizes the amount of data stored on the media, while providing the ability to recover from errors, access data randomly as needed, and do all of it quickly.

File systems, or more accurately, file storage systems, keep track of all the information about files stored on a disk, including in which sectors the data is stored. Rather than track one sector at a time, however, most file systems group multiple sectors together in what are called clusters.

Clusters are simply groups of 1, 2, 4, 8, 16 or more adjacent sectors2. A file system then tracks the location of a file’s data by keeping a list of the clusters assigned to it.

Running CHKDSK (no parameters required) will display the cluster size used on a drive as “bytes in each allocation unit” at the end of its report.

You can see that my hard disk has 4096 bytes per cluster3.

Space given versus space used

Conceptually, when I created my one-byte file, the file system had to do a few things:

• Create an entry in its table of files, or the “directory listing”, as it’s more commonly known.
• Allocate a cluster on the hard disk in which to store the file.
• Write the data to disk.

The file was given a cluster — 4,096 bytes of disk space — even though the file is only one byte.

A one-byte file takes up 4KB of space because that’s how disk space is allocated: one cluster at a time. Should the file grow to 4,097 bytes in size, an additional cluster will be allocated; the 4,097-byte file will actually take up 8,192 bytes of disk space.

Depending on where you’re looking, either number might be reported.

But File Explorer showed 1KB, not 4

Note that I said “conceptually”, above. In reality, that’s not quite what happened.

A file system tracks more than just your file’s data. It also records its name, the list of clusters allocated, timestamps, attributes, permissions, and more. All that “meta-data” (data about your data) takes up disk space in the file’s directory listing.

In the NTFS file system directory listing, space is allocated one “chunk” at a time. Regardless of the actual amount of meta-data, the space it’s given grows 1,024 bytes at a time.

The optimization is simply this: if the file is small enough, and there’s enough space left over in the directory listing to hold the file’s data, it’s placed there instead of being allocated any clusters at all. In a sense, the file takes up no additional space on disk (zero clusters) beyond its directory listing.

When that happens, Windows File Explorer lists the size as 1KB — the size of the directory listing — rather than the size of the clusters allocated to the file.

It’s the same online, except different

It’s all the same online, in the sense that cloud storage services use hard disks just like you and I do. Those hard disks are formatted with file systems, and those file systems allocate space in various ways that probably behave much like I’ve just described. There’s a good bet that Microsoft’s OneDrive uses NTFS-formatted hard disks to hold your files.

It’s all different in the sense that none of that matters, and the hard disks are hidden from you completely. All OneDrive and other cloud storage providers show you are your files and their actual file sizes.

While it’s important for you to know how much space your files are consuming on the hard disks on your machine, that information is completely irrelevant online. It could even change as cloud storage providers transparently update their infrastructure, possibly moving your data from hard disks formatted one way to hard disks formatted some other way.

The result of all this? The pictures you have on your hard disk that take up 88 gigabytes of space might actually be only 65 gigabytes’ worth of actual data. Chalk up the difference to file system overhead, and the fact that full clusters are allocated even for the smallest files.

Podcast audio

Download (right-click, Save-As) (Duration: 6:49 — 6.3MB)

Subscribe: RSS