I look after an office with 20 PCs and a server, and we run weekly full
backups and daily incrementals. Backups are to a USB-attached hard disk, which
is taken off-site overnight. What I’d ideally like to do is carry out the
backup process over the internet, to a remote PC with the backup drive
permanently attached to it, to avoid physically transporting the drive. Full
backup is around 120GB of data, incremental is 5 – 10GB. What would you
recommend as the best method to achieve this? (using an internet hosted,
paid-for backup service is not an option – too expensive!!)
On-line or internet hosted backup services (the ones you’re avoiding for the
cost in your situation) are becoming very popular. They definitely have their
place, but they also make me uncomfortable.
And they make me uncomfortable for the same reasons and issues that you’re
going to run into with what you’re attempting to do.
Become a Patron of Ask Leo! and go ad-free!
The problem you’ll run into is bandwidth.
Unless you have an incredible internet connection, the copy off-site will
take longer than the backup period. In the table below I’ve done a little
back-of-the-envelope math, and as you can see, a 120 gigabyte backup will take
over a week to upload at “traditional” T-1 or good DSL speeds. If you have a
VERY good internet connection – beyond the maximum of DSL rates, you’re still
talking a day to upload and that’s assuming perfect communications and 100% use
of the internet connection.
| Backup Amount | 1.5Mbs (T-1 or “good” DSL) | 10Mbs (basic ethernet LAN speeds) |
| 120gig | 7.5 days | 1.1 days |
| 10gig | 15.2 hours | 2.2 hours |
In my opinion across-internet types of backups really only make sense
if:
-
you have an extremely fast internet connection
-
or the amount of data that you’re backing up can somehow be constrained to a
more reasonable amount.
In other words, across-internet backups just don’t make sense for full
backups, and they rarely make sense in incremental backups in an busy
environment.
full backups, and they rarely make sense in incremental backups in a busy
environment.”
So what about those online backup services?
First, I’m sure that they’re doing everything as smart as they possibly can
to minimize the impact of bandwidth limits. I’m sure they compress whatever
they can, and that they copy only things that have changed, and that they use
your internet connection at what would otherwise be “idle” times – which for
many people is most of the time.
However, that doesn’t change the fact that a full backup is a heck
of a lot of data; much more than one would want to upload via any typical
internet connection.
So compromises must be made, and it’s those compromises that concern me.
The most typical compromise is to use an internet backup service to backup
only your data. This makes total sense, and is a great compromise, as long as
one huge issue is addressed. In fact, I do something very similar myself on
a semi-regular basis.
That one huge issue? How do you backup the rest of your system?
It’s an issue that’s very common when you choose to backup only your data,
regardless of the reason. The issue is simply this: what happens when you lose
something that isn’t part of what you backed up? An installed application,
perhaps, or even Windows itself? Your on-line data-only backup will not
help.
That’s why I recommend on-line backups only as part of a larger
backup strategy that includes full backups to more traditional backup
media.
To put it even more concretely, here’s what I do:
-
Every night important data is collected on one central machine. The other
machines are not backed up in any other way. This implies that if something
worst-case happens to them I will end up having to rebuild them from scratch.
Based on what they do and how they are used, this is an explicit choice I’ve
thought through. -
That central machine is actually my primary desktop machine. Full backups
are performed on it monthly, and incremental backups are taken nightly. I use
Macrium Reflect and back up to a NAS device on my network. -
Also nightly, some particularly critical data is copied across the
internet to the computers at my wife’s business, and vice versa. Because of the
bandwidth issues this is a limited amount of data; something that can be copied
in just a few hours in the middle of the night. (And the primary computer there
is also doing monthly full and nightly incremental backups using
Acronis to an external hard drive.) -
Once a month a larger collection of data is assembled, compressed and
encrypted, and uploaded to an off-site server. This is about 2.5 gigabytes of
data, and takes several hours to upload.
As you can see, remote backup can be a part, but only a part, of a
larger backup strategy. I strongly recommend off-site backup of some sort, and
it’s one approach to that. However, that doesn’t remove the need to do an
explicit and well-thought out backup of full systems and data.
Are you aware of any Windows backup utilities that include “bare metal” restore capabilities? By that, I mean the ability to recover after a hard drive crash, without first installing Windows.
—–BEGIN PGP SIGNED MESSAGE—–
Hash: SHA1
Most backup programs should support this. I know that
Acronis TrueImage will. It allows you to create a boot disk
that you can boot from and then restore an image to a hard
disk.
Leo
—–BEGIN PGP SIGNATURE—–
Version: GnuPG v1.4.7 (MingW32)
iD8DBQFHp+bhCMEe9B/8oqERAp6mAJ9UyNgJK3KftSiDgDQYr4a6Og15bgCfTuFv
7eazt38RnedILLbzhhtERgE=
=soLq
—–END PGP SIGNATURE—–
I strongly disagree that online backups don’t make sense. I was initially highly skeptical, but you’d be surprised how well over the wire backups can work even over relatively slow connections. I use an enterprise-class data replication package to backup several servers to a remote disaster recovery facility on just a T1. I’ve seen file servers with hundreds of active users replicate changes in real time using less than 512 Kb. Not heavy usage with lots of huge files, your typical office with MS Office files, PDFs, etc.
First, they’re smart about replicating changes. When a file changes, usually not every single bit of that file is changed. Only the changed bits are replicated.
Second, the data is compressed before being sent over the wire. Depending on the type of data, this can make a huge difference. Not with already-compressed formats like mp3, jpg, zip, etc. but significant reduction for MS Office docs, text files, PDFs, and many other file types.
The initial backup takes a long time, though compression will make the numbers better than stated above, possibly significantly better depending on the file types.
Online backup services generally aren’t cheap, but they’re convenient and they work. There are some pretty low cost options available for single PC backups, though enterprise-class is another story.
For the few who work with a lot of very large files or have a couple hundred GB of data or more, it’s probably not a reasonable option for all your data. Even in that circumstance, I wouldn’t discount the ability to use online backup for some of your files. Maybe not suitable for your entire MP3 catalog, but very suitable for your My Documents folder.
Then there are the questions of security, reliability and sustainability.
How secure is your data in the on-line location? Is your hosting solution trustworthy? Will they take the steps necessary to keep the rest of the world from seeing your data? Would it be a problem if the hosting company saw your data? What can you do to secure that information? Placing your data into an encrypted virtual disk, such as TruCrypt, would certainly protect your data, but you wouldn’t be able to perform incremental backups. Be cautious of other “secure” solutions, such as encrypted ZIP files, which may be easily decrypted.
Will your hosting solution be consistently available when your system is ready to backup? If you’re only sending large packets to an online service once a week, you probably want that service to be available when your packet is ready to send.
Finally, will the backup host be there when you need them most. Choose a solution that will stand the test of time and not quickly disappear (with your data) as so many Internet solutions have.
I’m working on an offsite solution, but won’t implement it until I get 15/15 mbps FIOS at the house. Then I’ll use TruCrypt to build encrypted volumes that I will then store on my web site hosting solution in a secure FTP folder. I won’t store my drive image – just my docs, photos, and music.
Hello there,
i’m using image software(acronis) for many years now, and i experienced that this type of back up is not working 100%. in some cases the image file on a second hard drive get’s damaged for some reason. I realy would like to know the reasons so a back up gets cocksure.
Every good online backup solution will encrypt the data before it’s sent, using a key stored on the local machine only. This means if the provider is compromised, the data is useless (assuming your key is strong enough to make cracking impractical – in actuality impossible unless whoever wants the data has access to millions of dollars of computing resources to successfully brute force your key).
This also means your key is critically important, because you can’t recover your own data without it. I recommend keeping a copy of it offsite somewhere, like a safe deposit box at your bank. And use something randomly generated and 10+ characters long including upper and lower case letters, numbers and other characters.
For full image backups, I use external hard drives that I rotate from a bank safe deposit box every 2-3 months or so. In the case of a significant disaster like a fire that destroys my data, a 2-3 month or even older full image backup isn’t a problem to restore and quickly get back to the previous state of your PC. I use Backup Exec System Recovery because it lets you restore to any hardware, with most imaging solutions you must restore to identical or near-identical hardware otherwise Windows won’t work.
Other data is far more critical, like the things people commonly keep in their My Documents folder. That is where online backups are an excellent solution because you aren’t exposed to significant data loss. No home or home office user is going to consistently make the effort to get data backed up offsite on a daily basis.
Given my advocating of online backup solutions, it might seem like I have a stake in them, but that’s not the case at all. :) I just know it’s the only way many home and small office users will ever get backups off-site on a timely basis, and I’ve seen too many times what happens when a solid backup strategy isn’t in place.
I too faced the same problem – same data size. The solution I deviced is as follows:
1. Made a mirror of Primary site on the secondary site.
2. Synched both the sites overnight. Since only changed data is transfered, time taken and bandwidth used is reasonable amount.
3. Run same backup setup on both the sites to create two identical off-site sets.
4. I am using VPN connection between the two sites to ensure data security over Internet.
The key points are Mirror and VPN. Rest is routine.
– Rahul.
Another option is on-site NAS if located in a secure space which is not effected by fire, flood, etc. It would be many feet from the PCs but connected by Ethernet cable to the same networks.
Then use incremental image or file data only backups nightly. NAS use any size HDDs and include multi-disk RAID capability. They also include USB connection so can make DVD or HDD copy to take off-site at selected intervals. Writing to USB does not effect the connected PCs because the NAS has its own OS.
Leo, I’ve taken your advice and recently invested in Acronis True Image Home (2010). I’ve reviewed your recommendations and videos concerning ATIH, but I haven’t noticed any references to the “Nonstop Backup” feature: it initially makes a full system/disk/partition backup to an external (USB) hard drive and then keeps adding to it incrementally (every 5 minutes) and continually consolidating all increments into one backup entity for as long as your PC is powered up.
A couple of weeks ago I had a full-system failure and by using the ATIH recovery feature I was back up and running in about an hour, _exactly_ to where I was just before the problem occurred. If I had been using the usual daily incremental scheme, I would have lost what I had been doing in the period between the previous incremental and the present.
Could you please comment on ATIH’s Nonstop Backup feature? For example, is there any reason not to use it? Thank you.
Frank D
01-Feb-2011
Hi Leo,
You say in your article
“Once a month a larger collection of data is assembled, compressed and encrypted, and uploaded to an off-site server”
What software do you use to upload to to the offsite server??
Regards
Peter Mclaughlin
01-Feb-2011
Hi,
Thanks for the answer on rsync.
Do you use the windows or unix version?
Regards
Peter Mc:aughlin
07-Feb-2011