They’re about safety.
When you go to a download website to download a large file, you might see a series of numbers displayed alongside the download link. These are commonly referred to as checksums or hashes and allow you to confirm that the download completed successfully.
The example above for Hiren’s Boot CD PE shows a link for what some would consider a fairly large download: over three gigabytes. (If you’re curious, Hiren’s Boot CD provides diagnostic and recovery tools that run from a bootable CD, DVD, or USB drive.) We’ll use it as an example as we look at why checksums exist and how to use them.
Checking checksums
Checksums like MD5 and SHA are long numbers that prove a file you downloaded is complete and unchanged. By comparing the site’s published checksum with one you calculate, you can confirm the file isn’t broken or tampered with. It provides extra safety for important downloads.
Fragile downloads
The use of checksums has evolved. In years past, even if a large download completed, it was not unheard of for portions to have been garbled along the way1. Running a tool to calculate the checksum on what you just downloaded and comparing it to the expected checksum allowed you to determine if you’d received everything as it should be.
These days, all you really need is patience. Depending on your internet speed, you can generally count on the download eventually completing without incident.
Help keep it going by becoming a Patron.
Related
How Can I Tell If a Download is Safe?
Checking whether a download is safe before you download it is nearly impossible. Your best defense is your own skepticism, plus anti-malware tools to scan what you choose to download.#74029
Malicious downloads
A more recent use for checksums is to confirm that what you downloaded is what was actually provided and not some malicious imposter. Again, calculating the checksum on what you downloaded and comparing it to what you expected told you whether you’d received what you intended to download.
It’s not perfect (someone capable of placing a malicious download could potentially update the checksums to match), but there are scenarios where it still adds value. For example, on mirror sites — multiple sites that share the work of hosting downloads — the official source publishes the checksums. If a single mirror is compromised by a hacker, it can be detected when the checksums are compared.
Checksums
Checksums — or more correctly, hash values — are mathematical operations that read the entire file and generate a large number based on its contents. Even a tiny (one bit!) change in the original file results in a dramatically different number, meaning they’re great for change detection. If two files generate the exact same hash, we know they’re the exact same file.2
So, let’s say you have downloaded this version of Hiren’s Boot CD PE.
First, you right-click on the file and choose Properties so you can check the filename, “HBCD_PE_x64.iso”, and the exact file size: 3,291,686,912 bytes.
Next, we calculate checksums using one or more of the listed hashing algorithms: MD5, SHA1, or SHA-256. Any will do, but I’ll do all three.
While I’m sure there are plenty of third-party Windows tools you can find to do this, you don’t need them. Everything you need is available in the Windows Command Prompt.
The magic command is:
certutil -hashfile <filename> <hash algorithm>
To run the MD5 hash on the file we downloaded, we would type:
certutil -hashfile HBCD_PE_x64.iso MD5
Here are the results of calculating each of the three hashes provided.
Compared to the checksums in the image at the top of this article, the calculated values match.
MD5 – 45baab64b088431bdf3370292e9a74b0
SHA1 – a0cdff7c5ec8b1c6dade4b5b55068cffca545318
SHA-256 – 8c4c670c9c84d6c4b5a9c32e0aa5a55d8c23de851d259207d54679ea774c2498
We got the file we expected.
Technically, SHA1 and SHA-256 are more robust and secure than MD5. In practice, though, MD5 is sufficient for simple file comparisons and is usually faster.3
Related
Is It Safe to Download from Download Sites?
Download sites are just too risky these days. Don't use them unless there is no way to avoid them, and even then, be very careful in your selections.#13546
If the hashes don’t match
When checksums don’t match, life gets interesting.
My first approach is to download the file again in case there was a problem with the download. Ideally, I might do this on another machine if I have one available.4
If the hashes still don’t match, then:
- If both your downloads calculate the same hash, but it doesn’t match the download site’s hash, then there’s likely a problem with the source file. Either it’s been changed, or the hashes haven’t been updated to match. If you’re downloading from a mirror, choose a different one. Otherwise, I’d probably reach out to the file’s provider to see if they’re aware of anything amiss.
- If your downloads calculate a different hash and both are wrong, that points to an issue with the download process. Not only would I try a different machine, but I might try a different location or internet connection. This is rare.
Most of the time, hashes match.
Hashes are for more than downloads
Hashes are great for tracking file changes — or rather, lack of change — in general. For example, a duplicate file finder might cache hashes on files it scans so as not to need to scan them again every time you run it.
I’ve used hashes across machines. If I want to see if the file on machine A is exactly the same as that on machine B, there are two methods I could use:
- Copy one file to the other machine and compare the two.
- Calculate a hash on each machine and compare the results.
Reading the file and calculating the hash is typically much faster than copying something across a network.
Do this
I rarely take the time to validate download checksums. Things are reliable these days, and the sites I download from are well-established and secure. However, if there’s any question, now you have a tool that confirms your downloads, as long as the download site supplies the expected values.
Subscribe to Confident Computing! Less frustration and more confidence, solutions, answers, and tips in your inbox every week.
Podcast audio
Footnotes & References
1: Among other things, download protocols — the methods used to transfer files from one place to another — have improved dramatically, as has the underlying internet connection; we’re not using noisy dial-up modems anymore.
2: Statistically speaking. While two different files can generate the same hash value, it is so statistically improbable as to be a non-issue.
3: The difference matters much more when hashes are used in encryption, which is another common application.
4: Bonus points if you can use a different OS, such as Linux or Mac, which includes similar tools, just to rule out any Windows-specific issues. It’s unlikely, but if we’re ruling things out, this is an easy one if you have another computer lying about.
When you download a file, most protocols include a CRC or other checksum to verify that the file was transmitted correctly.
It’s not as strong as a cryptographic hash (like SHA-256), but it’s “good enough” to confirm that the file arrived intact. That’s why we can confidently download a file and expect it to be intact.
Right. The provided checksums these days are less about transmission errors, caught by the download protocols, but other issues that might arrise.