Considering New Backup Options

[Greg V]

My backup process is to use Mac OSX Time Machine daily (hourly when I happen to be using the computer) to backup my iMac hard drive to a networked hard drive. Weekly I clone my iMac hard drive to a different portable hard drive using Carbon Copy Cloner for a bootable backup. Also on a weekly schedule is a backup of my important personal and financial records to a flash drive.

I don't use Dropbox in the backup scheme, but find it very useful for data exchange among my desktop and mobile devices in support of specific apps.

There are many good ways to handle backups...it's just important to have a system and use it consistently.

[sengsational]

This might be considered a "plug", but I really think it would save a huge percent of 'type 1' problems: Spinrite. This program is not free, but if it doesn't live up to your satisfaction, you can get your money back. The program has been around since the days of the IBM PC, but has been updated along the way, of course.

Current drives have a system called SMART that moves data to new sectors if it takes 'too many' read attempts to get the data. The problem sector is marked so its out of commission. One function of this program is to read every sector on the drive, which lets the SMART system do its magic. Obviously you could have a drive that goes TU all of the sudden, but I have found that they often start going slowly (evidenced by longer and longer read times). Spinrite has a mode where it will, non-destructively re-write every sector too. Works on any file system (even TIVO drives), but you need to get the drive mounted on a box that can boot DOS.

[imoldernu]

Imoldernu's files are preserved on this computer, to be recovered at a later date in a landfill, and placed in my memorial library, where they will rest untouched and unread by anyone.

Hudreds of thousands of files, pictures, essays, music and once treasured memories will disappear when the computer gives up the ghost. R.I.P.

The few important documents are preserved on Google Drive and on an 8G thumb drive. The vanity of my avatar is no longer a goal. Memory is transient, physically, mechanically and electronically. Better forgotten, than to become a guilt object for those who follow.

[Tractor guy]

I use SecondCopy (available here: Automatic backup software - secure data with Second Copy ) to backup pictures and data files to a 1T NAS drive that is sitting on my home network. It runs in the background and does incremental backups once a week. I can also kick off the process on demand if needed. I do this for example if I'm going to take my laptop on vacation. I make sure that everything is backed up before I leave. I've got it set so that it keeps one old copy of a file whenever it updates the backup.

This takes care of cases 1 and 3 in the OP list of bad things that can happen. (disk failure or accidental deletion). It doesn't take care of the fire or tornado risk. I've been thinking about using Carbonite for that but haven't got off my duff yet.

I've been using Second copy for about 12 years now. It got great reviews when I paid for it but seems to have disappeared from people's radar since then.

[KingB]

I try to keep it simple. I use Windows 8.1's File History. I have a portable HD that is always connected to the desktop's USB3.0

[mpeirce]

Quote:

Originally Posted by photoguy I definitely have several GB of files that could be deleted. But given how cheap memory storage is, I've decided it wasn't worth the time to figure out what needs to be deleted and instead keep everything.

I too err on the side of keeping stuff around. But I use this too:

GrandPerspective


It looks over your disk and generates a nice graphical map of your files showing their sizes. It's very useful in finding giant files that I really don't need any more (movies, disk images, etc.). This is for Mac, but I'm sure there is something similar for the PC out there.



For backups I use TimeMachine. I have a local disk that I keep connected to my iMac and periodically connect to my MacBook. I also have another large disk that I keep in my safe deposit box that I fetch every month or two and let TimeMachine do its thing, then returning it to the safe deposit box.

This way I have a catastrophic backup that's fairly up to date and continuous backups for the dumb mistakes or hardware failures.

[gauss]

Quote:

Originally Posted by mpeirce I too err on the side of keeping stuff around. But I use this too: GrandPerspective It looks over your disk and generates a nice graphical map of your files showing their sizes. It's very useful in finding giant files that I really don't need any more (movies, disk images, etc.). This is for Mac, but I'm sure there is something similar for the PC out there. .

Indeed! I use WinDirStat for Windows machines.

-gauss

[TromboneAl]

I just found out that OneDrive doesn't allow any files over 2 GB. That ruins my backup scheme, unless I have my backup app split the files.

[target2019]

Quote:

Originally Posted by TromboneAl I just found out that OneDrive doesn't allow any files over 2 GB. That ruins my backup scheme, unless I have my backup app split the files.

Googledrive had a 10GB filesize limit, but it is now 1TB.
I have 15GB in a free gmail account I just checked. I think that may work for you...

Don't use it for that purpose currently, as I prefer local net drive.

[TromboneAl]

Quote:

Originally Posted by target2019 Googledrive had a 10GB filesize limit, but it is now 1TB. I have 15GB in a free gmail account I just checked. I think that may work for you... Don't use it for that purpose currently, as I prefer local net drive.

Yes, I just integrated my googledrive with File Explorer, but I haven't looked into whether it is duplicating the files on my hard drive or not.

I'd need more than the 15 free GBytes that I currently have to use that for backup.

I get around the 2GByte limit on file size on OneDrive by telling the backup app to split the files. Works OK so far.

[target2019]

Quote:

Originally Posted by TromboneAl Yes, I just integrated my googledrive with File Explorer, but I haven't looked into whether it is duplicating the files on my hard drive or not. I'd need more than the 15 free GBytes that I currently have to use that for backup. I get around the 2GByte limit on file size on OneDrive by telling the backup app to split the files. Works OK so far.

Gotcha. I don't use gdrive for backup. I don't auto-synch entire My Documents folder to a cloud.

In My Documents folder, I have a folder "Google Drive" and I copy files to that which I need accessible on other devices. I thought you might try to copy your entire 2GB backup file into a folder that is synched automatically with their cloud. Could get confusing, though. If you copy such a thing to a shared folder, you'd want to exclude that from your local backup process.

Interesting how there are so many approaches to skinning this backup problem.

[Meadbh]

Quote:

Originally Posted by mpeirce I too err on the side of keeping stuff around. But I use this too: GrandPerspective It looks over your disk and generates a nice graphical map of your files showing their sizes. It's very useful in finding giant files that I really don't need any more (movies, disk images, etc.). This is for Mac, but I'm sure there is something similar for the PC out there. For backups I use TimeMachine. I have a local disk that I keep connected to my iMac and periodically connect to my MacBook. I also have another large disk that I keep in my safe deposit box that I fetch every month or two and let TimeMachine do its thing, then returning it to the safe deposit box. This way I have a catastrophic backup that's fairly up to date and continuous backups for the dumb mistakes or hardware failures.

It would make a nice quilt, too!

[TromboneAl]

Here's an update:

I have my computer do a backup of the documents folder every Friday at 3 AM. It backs up to cloud storage, and it happens automatically. Getting the task scheduler to do this was a bear, because of all the gotchas. Some of the reasons it didn't work: Set to run only when plugged in, exe file has to be set to run as administrator, you must allow wake timers in the power options.

Each of those things had to be solved individually (via Googling), and there was no indication of why it wouldn't trigger.

But it will be nice to have the backups done unattended.

[NW-Bound]

I am no longer that patient to solve this kind of riddles. Or perhaps I do not care that much anymore to fight with the OS.

As stated earlier, I have a PC converted to a home server. I can turn it on remotely from my laptop, which roams on WiFi. Then, I initiate the backup manually whenever I remember about it.

The above is a full Windows back up. In between, I only have the Quicken file, a few data files, and new photos to backup. These, I backup to another standalone NAS server running 24/7. Quicken is directed to this NAS for its auto backup. This small server shuts down its hard drive during inactivity, so I do not feel bad about leaving it on.

I am still susceptible to the "big castastrophe", and need to think about storing on the cloud just the critical financial records. The bulk of the media files such as photos and MP3s are too big to put on the cloud, and will go on a USB drive placed in my motorhome.

[ERD50]

Quote:

Originally Posted by TromboneAl Here's an update: I have my computer do a backup of the documents folder every Friday at 3 AM. It backs up to cloud storage, and it happens automatically. Getting the task scheduler to do this was a bear, because of all the gotchas. Some of the reasons it didn't work: Set to run only when plugged in, exe file has to be set to run as administrator, you must allow wake timers in the power options. Each of those things had to be solved individually (via Googling), and there was no indication of why it wouldn't trigger. But it will be nice to have the backups done unattended.

Something that would be far simpler, and almost as good, would to be to just set a pop-up reminder on Fridays. If you had this down to a one button operation while you are powered, that would be almost as good. That would eliminate all the complexities/gotchas of waking up and running a script unattended.

Now, how do you verify that it worked? Is that automated? How do you verify that?

My motto is -If you have not successfully restored from the back-up, you do not have a back-up.

I do it manually (and not often enough, so it is far from perfect), but I do open at least a few files on the backup for a sanity check.

So here's a question for the techies:

I see that others have said they have too much to put to the cloud. Here is what I was thinking - I do several full backups of my 'stuff' occasionally (I treat music, photos and videos separately), then I do incremental updates to those bases, until I decide to delete one and start a new base. Now, here is what could be handy for 'the cloud':

A) Make a full baseline back-up to several portable drives.
B) Do a full incremental backup to the cloud, but use the directory listing on the portable drive as a reference for the incremental backup.

This way, only the incremental stuff would be in the cloud.

I kind of did this a while back - I wrote some scripts that would back up files that were X days old (prompted for X), and it retained the directory structure. I had that set up to zip and store to USB, but I could redirect that to the cloud. But if you do this once per day, you end up with a bunch of files and you would need to do some searching to find something. Hmmmm .... OK, picture this as a daily routine for simplicity:

A) First run my 'recently changed' script to capture all the files that were changed in the one day since I backed up to portable drives, and store to the cloud unzipped.

B) Run my 'recently changed' script every day after that and capture files changed only within the last day (plus a few hours for overlap) - copy these temporarily to a directory on my hard drive.

C) Now do an incremental update to the cloud from that temp directory. Delete the temp directory. Repeat from 'B'.

That should build up all the changed files in one neat directory structure in the cloud. Baseline would be missing, but I have that on my backups. And I could sync from the cloud to my backups if my internal HD crashed, and have everything in one place. Assuming rsync works to the cloud, but I'm pretty sure it does.

Since only the daily changes go to the cloud, this should be fast.

Anyone see any problems? This should be easy to turn into a one or two button script.

-ERD50

[TromboneAl]

The problem with the manual system is you are using the computer, so the machine will run slower. So then you think, "I'll finish what I'm doing, then do the backup," and it often doesn't happen. I used that system for years, and often the Friday backup didn't happen.

It's quick to confirm that the new file exists, and If the app runs, it will be on my screen in the AM, so I'll know it ran.

The problem with the incremental system is that you aren't protected from a hard drive crash.

My auto backup takes only 7 minutes, and 20 of them will fit on the cloud drive.

Sent from my Nexus 7 using Early Retirement Forum mobile app

[ERD50]

Quote:

Originally Posted by TromboneAl The problem with the manual system is you are using the computer, so the machine will run slower. So then you think, "I'll finish what I'm doing, then do the backup," and it often doesn't happen. I used that system for years, and often the Friday backup didn't happen. It's quick to confirm that the new file exists, and If the app runs, it will be on my screen in the AM, so I'll know it ran. The problem with the incremental system is that you aren't protected from a hard drive crash. My auto backup takes only 7 minutes, and 20 of them will fit on the cloud drive. Sent from my Nexus 7 using Early Retirement Forum mobile app

A daily backup is really, really quick, and doesn't bog down the machine (yes, will slow disk access a bit, for less than a minute probably). Rsync uses low level commands (grsync provides a GUI if you prefer), it is fast and efficient.

Why am I not protected from a hard drive crash with incremental? That's not true. I have a baseline complete backup on multiple portable drives, the incremental simply adds any changed or new files to that baseline.

OK, if you are talking about the system being backed up, yes, this is for data. I use some other strategies for system backup. But that isn't an incremental versus full back up issue, it's a system-wide versus data-only issue.

-ERD50

[ERD50]

Quote:

Originally Posted by TromboneAl The problem with the manual system is ....

Just to be clear, I'm not trying to say one way is better than the other (manual/auto). They each have pros/cons, people should choose whatever works for them.

-ERD50

[TromboneAl]

Quote:

Originally Posted by ERD50 Why am I not protected from a hard drive crash with incremental? That's not true. I have a baseline complete backup on multiple portable drives, the incremental simply adds any changed or new files to that baseline.

Sorry, I was at starbucks, read quickly, and thought you said that the first, full backup was on the main drive.

Incrementals are amazing for saving space, but when you are stressed about finding a particular file, it's nice to just have one single, simple location for that file.

Note that File History on Windows 8.1 also automatically backs up files full time (backing up each time you save one).

[ERD50]

Quote:

Originally Posted by TromboneAl ... Incrementals are amazing for saving space, but when you are stressed about finding a particular file, it's nice to just have one single, simple location for that file. ...

I use rsync/grsync - it doesn't work that way (or I would not use it!). My backups look exactly like my source.

When you run it, it does a compare from SOURCE to DESTINATION, if the file in the DESTINATION is older (or non-existent), it over-writes (or copies) it. So everything is in one place, the DESTINATION looks exactly like your SOURCE. You can configure it differently, but I have it not delete files that are in the DESTINATION but not the SOURCE. So if I accidentally delete something, it will be on my backup. But over time, your DESTINATION can get larger than your SOURCE (but not by a lot, unless you change a large directory structure or something, which could make everything under it look 'new').

edit/add - OK, if I were to put the incrementals on the cloud, yes, I'd need to merge them with the baseline on my portable drive to have them all in one place. Currently though, I do the incremental to my portable drives, so everything is in one place. Considering how few times (probably never) that I'd need to go to the cloud incrementals, I'm not too worried about that, I'll still update my portables regularly.

Quote:

Note that File History on Windows 8.1 also automatically backs up files full time (backing up each time you save one).

Nice, but I'm always nervous about a backup to the same drive (I'm assuming here), but great protection against the accidental delete or oooops!

-ERD50