Technology in terms you understand. Sign up for the Confident Computing newsletter for weekly solutions to make your life easier. Click here and get The Ask Leo! Guide to Staying Safe on the Internet — FREE Edition as my thank you for subscribing!

Why am I seeing these odd defragmenting results and why is it taking so long?

Question:

I’ve read your article which likens defragging to picking up the bits of a
windblown newspaper from the garden and how badly a teenager might do this, but
strange things happen when defragging an external hard drive. On a 150 GB drive
with 73 GB free, I start with just one fragmented file, a 47.7 MB in two
fragments. After Defraggler has run for two hours and 10 minutes, I still have
one fragmented file of 47.7 MB, but now in three fragments. That’s an
improvement? On a 300 GB, with 75% free, I start with 15,000 files fragmented
in 45.7 GB. In 72,000 fragments and 12 hours later with 46% defragged, I still
have 8,000 fragmented files, of 30 GB in 44,000 fragments with a calculated
time remaining of 7 hours and 10 minutes. Ultimately, why the discrepancies in
the end result in the first instance and the horrendous amount of time it takes
in the second instance? Especially here in the second instance where the system
has 75% free space to copy files to join up the fragments and move them to the
beginning of the filed drive space?

In this excerpt from
Answercast #67
, I look at a case where the results from defragging an
external drive don’t seem to make sense.

Become a Patron of Ask Leo! and go ad-free!

Defragmenting files

OK, there are several things at play here.

First let’s talk about that first case where a file has only two
fragments when it starts and may end up having three fragments when it’s done.
I realize that is totally counterintuitive. That’s not at all what you
expect.

But your question of, “Is that an improvement?” You know, it could very well
be. And the reason may be that those three fragments are actually
closer to each other than the two fragments were that we started with.

Consider this scenario where you’ve got a large hard drive. This was a 150
GB drive and yes, it has lots of space free. Your file is in two fragments. The
worst-case scenario is that one fragment of the file is at very beginning of
the disk and the second fragment of the file is at the very end of the disk.
So, reading that file could potentially take a fair amount of time – where it
would at least have this big hiccup in the middle of reading it, where the
disk head has to move from the entire front end of the disk to the back end of
the disk.

Defragging moves segments closer

Now, let’s defrag it. The defragger may prioritize getting the file closer
together in that kind of an extreme case. So the result is having three
fragments, but those three fragments are all physically very close to each
other, say at the beginning of the disk. That could indeed be a significant
speed improvement.

Like I said, it’s not intuitive and it’s part of the reason that different
defragmenting programs will give you different kinds of results. I’m not
familiar with how Defraggler prioritizes things. I’m just kind of speculating
here that yes, this is a scenario where a file of three fragments could be
perceived as a calculated improvement over a file with only two fragments;
depending on where those fragments happen to reside on the hard disk media.

Defragging takes time

Now, let’s look at this other one: the 300 GB drive again with 75% free.
You’ve got a bunch of files; it’s taking a really long time.

You know, there are two things here that are very difficult to conceptualize.
One is defragging is an awful lot of work – especially if the files are highly
fragmented to begin with. It doesn’t really matter that there’s a bunch of free
space. I mean it matters. But it doesn’t matter in the sense that it’s not
going to make a tremendous amount of difference to (as you say) collect the
pieces of the file into a single file that’s in the free space and then copy
that single file back to a larger space elsewhere on the disk.

Realize that the fragmentation can be so bad that there may not be a
contiguous piece of free space big enough for the file that you’re
defragmenting. So what that means is that the defragger actually has to move
things around in order to make space (even though there’s lots of free space)
in order to make contiguous free space for the defragging operation to
work.

How defragging works

Now, some work that way. They will actually defragment the free space and
then use the size that it has collected in contiguous free space to reassemble
files and then copy them back.

Most will actually do the opposite. Most will typically start moving the
files around, if you will, in order to make a contiguous piece of space where
it can then assemble all of the pieces of the file with the rest of the file.
So it’s not really working by assembling the files in free space. What it’s
really doing is moving things around so as to make contiguous areas in
what will eventually be the used space.

Like I said, it is complicated. That can involve a tremendous amount of
copying. And by tremendous, I honestly mean tremendous. A highly fragmented
disk can cause a lot of activity as all of these file pieces are moved around
to make room for the reassembly of all these other file pieces into hopefully
what will result in a more defragmented disk.

External drives are slower

Now the other thing that you’ve got going against you here is that you’ve
got an external hard drive. External hard drives unfortunately are slower than
your internal ones. At least USB 2 and USB 1 [are] significantly slower.

So, defragmenting an external hard disk on a USB 2 connection is definitely
going to take significantly more time than defragmenting the equivalent disk on
an internal connection.

I don’t know about USB 3; I honestly don’t. I just touched on USB 3 myself
for the first time within this last week and my initial thinking is that it may
be pretty darned fast. But since most people are still running USB 2 for
external hard drives, I think that’s what is also contributing to this.

So ultimately, it’s very difficult to underestimate the amount of
file fragment moving that goes on when a defragging program does its job.

Different defragging programs

Exactly how a defragmenter defrags? This is one of the things that
differentiates one program from another.

It differentiates Defraggler from Microsoft’s Windows’ own internal
defragging program. They go about it differently. One may take longer or
shorter than the other. They may result in better or more defragmented disks
when they are done. Sometimes, they don’t.

Multiple passes

One of the things that is sometimes interesting to do is to defrag a disk,
and then defrag it again right away.

You will almost always find that there’s still some defragging that can go
on the second time. The first time didn’t get it perfect. That’s because
there’s just so much work involved; it is extremely difficult to get it right
without doing, like I said, multiple passes in a case like that.

Increased speed

The good news is that what you’re really going for with defragging is
speed.

You don’t have to be perfect to get the lion’s share of the speed
improvement from defragging. So defragging once, defragging most everything,
still having some fragments left over, is fine. Your hard disk will definitely
be faster than it used to be if you started with a highly fragmented hard
disk.

Ultimately, yes, defragging is kind of a black art. It’s very disk intensive
and it is (like I said) very difficult to overestimate the amount of activity
that it really takes to defrag a hard disk.

Do this

Subscribe to Confident Computing! Less frustration and more confidence, solutions, answers, and tips in your inbox every week.

I'll see you there!

8 comments on “Why am I seeing these odd defragmenting results and why is it taking so long?”

  1. One way to look at defragging, is a little like those tile sliding puzzles where you have something like 25 tiles and one free space and you have to move the tiles around in the frame to unscramble the picture or put the numbers in sequence. The defrag has more than one free space and it can move it to any free space not necessarily contiguous, but it has a lot more than 15 tiles to move.

    Reply
  2. Thank you Connie and Leo. The article puts defragging in a much clearer light and makes me realise that sometimes one has to use Edward de Bono’s lateral thinking rather than plain old logic.

    Very useful article.

    Reply
  3. Another way of thinking about de-fragging is is to compare with re-arranging a room almost full of furniture, without taking any out, whilst trying to finish up with logical association, eg dining table and chairs in a new location, other furniture arranged for easy TV viewing, doors and windows not blocked etc.

    For example, arranging to suit a party, then returning to normal family living afterwards.

    Watch my lady-wife tackling it!

    Reply
  4. I just ordered a new Dell XPS 8500 which comes with a standard 2TB SATA HD. I am only going to have about 50GB of programs and data on the PC. I shudder to think about how long its going to take to defrag this monster!

    Reply
  5. For the perfectionist, who wants that last fragmented file compacted into a single fragment, there is a trick that usually works, if you have adequate disk space. Simply copy the file, delete the original, and then rename the copy. On a recently (and thoroughly) defraged drive, this will usually produce a file with no fragments.

    Two tips for a cleaner, faster defrag operation: First, do a thorough disk cleanup (I use CCleaner) and EMPTY THE RECYCLE BIN. Second, Then restart Windows and run defrag on the newly restarted system You will save much fore time overall than the delay to restart.

    Reply
  6. After I close my browser for the day, I use C Cleaner then Auslogics Free defragger on both my external and C drives. Total time under 5 minutes. Only then do I shut down.

    Reply
  7. I ran Defraggler on a customer’s computer the other night. It was taking forever to complete (more than one day). As it runs you can see the file name as it moves things around. Very, very pokey. I just gave up and used My Defrag. This has always been a very highly acclaimed free program. I would estimate that the file names flew by at least three times as fast as Defraggler. It was done in a few hours.

    I can’t see beating an old computer to death with more than a day of defragging. This was only a 80 gig unit.

    Reply
  8. Thanks for this article. You helped me understand why my 1 terabyte external drive was taking almost 2 days to defrag, plus now I have more options for taking care of it. I LOVE your work!

    Reply

Leave a reply:

Before commenting please:

  • Read the article.
  • Comment on the article.
  • No personal information.
  • No spam.

Comments violating those rules will be removed. Comments that don't add value will be removed, including off-topic or content-free comments, or comments that look even a little bit like spam. All comments containing links and certain keywords will be moderated before publication.

I want comments to be valuable for everyone, including those who come later and take the time to read.