Hard lesson learned - Everythings gone

[Deleted member 21335]

Quick update. The software found a LOT of files on the time capsule hard disk that had been deleted, but looking at the previews, a lot of them are corrupt. There are THOUSANDS of JPG's and not way of distinguishing them without previewing every one which will take a short lifetime. I am really thinking of just writing the whole lot off and starting again. Not entirely sure how the other HD would get on, if they do a mechanical dismantle and repair. Would the files be as they were stored, or all corrupt and all over the show like the deleted ones?

 

[Hugh Jarse]

never ever ever ever ever carry out any hardware work whilst you have no backup in place..

I would agree with people saying to try and recover the erased time capsule.

disagree. DVD/CD delaminate over time if not stored correctly. if youre going to use optical look at bluray which is constructed differently.

Neil

Spot on

Blu ray is more robust in construction and in the accuracy of the laser. Hence 25Gb and 50Gb storage.

The discs still need proper storage and handling.

I have completed moving data on my oldest CDs to blu ray. I will keep my CDs but all back up is blu ray now.

S

 

[kendo1]

Quick update. The software found a LOT of files on the time capsule hard disk that had been deleted, but looking at the previews, a lot of them are corrupt. There are THOUSANDS of JPG's and not way of distinguishing them without previewing every one which will take a short lifetime. I am really thinking of just writing the whole lot off and starting again. Not entirely sure how the other HD would get on, if they do a mechanical dismantle and repair. Would the files be as they were stored, or all corrupt and all over the show like the deleted ones?

They should be okay as long as there has not been any physical damage.

Not all of your files on the time capsule will be corrupted, depends on several factors. Some will be 100%, others total garbage

Good luck, it's been a warning to us all.

 

[Pookeyhead]

I have completed moving data on my oldest CDs to blu ray. I will keep my CDs but all back up is blu ray now.

S

Are these files now only on optical disc with no other copies available? If so, they are not backed up, no matter what format the disc is. If these are the only copy you have, then they are archived, not backed up.

Back up means you have at least one copy somewhere else.

I once lost work that cost me at least £2000 a few years ago. I decided to invest in a decent back up system after that. The problem is that until this happens to you, back up never seems like a good investment, as you see no benefit... and hopefully, you never WILL see the benefit. People like to spend money on shiny things.. not something they never see. It's like insurance... no one likes buying that either.


Here's my regime:
I back up everything to a second internal drive (automated).
That is then backed up to my main RAID6 server (automated)
That server is then mirrored to another RIAD5 server in my garage (automated).

At any one time I have 4 copies of all files, two of those copies are on systems with redundancy. The main server can actually stand t have two consecutive disk failures and survive. In order to lose data the main storage drive in my main machine would have to die, then my first back up drive, then TWO drives in my main server, then another drive in my mirror. I'd need FOUR disk failures simultaneously... and even then I'd probably be able to recover most of it.

It sounds extravagant, but all in I probably spend less than £1200 on the system.

When drives are full I either replace or expand. When I first built the system it used 1.5TB drives to give 3TB of space.... it's since been expanded using 4TB drives to give me 8TB of space. Hopefully when space becomes an issue again, there will be bigger disks available... if not, then I'll have to invest in a redundant archiving system.

If you earn money with your data, then you have no choice but to invest in something that makes it secure. To lose data now is very unlikely, and would probaly mean something serious enough to take out the whole house, and garage (which is 15 metres away).... in which case I've got much bugger issues than data loss!


Sorry... I'm not preaching... just serious about back up.

 

[Almost_Invisible]

Sorry... I'm not preaching... just serious about back up.

Also not preaching or wanting to be the cause of additional outlay on hardware but ... what happens in the event of a fire?

 

[Deleted member 21335]

That's quite a backup system. I don't, nor will I ever earn any money from my photography. It's just not good enough. But I was surprised how bothered I am now it's all gone. Hard drive is currently with a data recovery chap but communication seems slow and I'm not confident to be honest.

 

[Steve Smith]

I'm glad I don't have to deal with all of this nonsense.

My routine:

1. Open box.
2. Check negatives are still there.
3. Close box.

I don't think there is a 100% safe method of storing anything. Even with multiple backups, I wonder if people are sometimes making perfect copies of already corrupt files.


Steve.

 

[hollis_f]

This is why I always recommend backing up to DVD/CD disk first

!!!!

I really do not have the patience to sit there and run 120 DVDs through my computer as it does a backup. In the same amount of time I can backup everything to a 1TB hard drive, backup everything to another 1TB hard drive, get in my car and take the second drive to my friend's office and put it in her fire safe, drive back home, backup everything to a third hard drive and put that in the annexe, set up an automated backup to my NAS, watch a couple of movies and sleep for 8 hours.

And where would I store 120 dvds?

 

[Mikesphotaes]

!!!!

I really do not have the patience to sit there and run 120 DVDs through my computer as it does a backup. In the same amount of time I can backup everything to a 1TB hard drive, backup everything to another 1TB hard drive, get in my car and take the second drive to my friend's office and put it in her fire safe, drive back home, backup everything to a third hard drive and put that in the annexe, set up an automated backup to my NAS, watch a couple of movies and sleep for 8 hours.

And where would I store 120 dvds?

My last line of backups is on DVDs and I do accept it would take a very long time to get them back on my PC, that's why it is my last line of defence.

120 in flexi-plastic sleeves takes up less than 10" depth, so you must have a really wee house!

I keep them in a drawer away from light and have absolutely NO problems reading discs, some of which are reaching 13 years old.

 

[BrynBoru]

neil_g quotes:-

disagree. DVD/CD delaminate over time if not stored correctly. if youre going to use optical look at bluray which is constructed differently.

Just noticed you've quoted me, though out of context. Safety is relative. I would never use optical media, of which blu-ray is the most prone to corruption.
Optical media is not a professional way to back-up large libraries of raw data. Hence my recommendation as written, which is at the low end of investment.

 

[M.A.Gibson]

I would suggest a reliable cheap NAS option is to purchase a small NAS i.e. D-Link which holds 2 drives stick 2 3tb drives in it (est cost - Enclosure £60 2x 3tb drives £200 http://www.amazon.co.uk/D-Link-ShareCenter-Network-Storage-Enclosure/dp/B004AIWOXY) and then backup to that on a regular basis that way atleast you're covered if the main machine dies.

 

[Deleted member 49549]

My backup regime

Current years pictures are on my local HD, previous years are on an external HD.

Both of these are backed up to a via 2 x time machine backups (ona at my house, one at my girlfriends)
I have a mirror of my external hd left at my fathers house. This gets replaced annually.
I also have a cloud backup on Amazon cloud.

So 4 backups. If i do lose anything it will be through something stupid i have done!

 

[Deleted member 49549]

I'm glad I don't have to deal with all of this nonsense.

My routine:

1. Open box.
2. Check negatives are still there.
3. Close box.

I don't think there is a 100% safe method of storing anything. Even with multiple backups, I wonder if people are sometimes making perfect copies of already corrupt files.


Steve.

A backup system with versioning may help with that.

 

[petersmart]

Neil

Spot on

Blu ray is more robust in construction and in the accuracy of the laser. Hence 25Gb and 50Gb storage.

The discs still need proper storage and handling.

I have completed moving data on my oldest CDs to blu ray. I will keep my CDs but all back up is blu ray now.

S

Unfortunately there are 2 types of Blu-Ray, HTL and LTH.

http://blog.digistor.com/not-all-blu-ray-discs-are-created-equal-but-does-bd-r-quality-matter/

So if you are going to keep them for a long time it appears that the HTL type are the ones to go for.

.

 

[Rojibo]

Rarely post, but I felt the need to weigh in here. I finally sorted out a decent backup solution for myself, as I'd been ignoring it and it would have been rather embarrassing to suffer data loss given my occupation.

Have you had a look at the cloud backup solutions? Alot of places are offering unlimited cloud backup for £x/month, generally you install a client and it performs and initial backup of everything, then it monitors your selected directories and updates files as they change, or as new ones are added.

Crashplan do a very good free trial that might be worth checking out, obviously the advantage here is you're combining your backup and DR solution in one. I went with a company called Squirrel save in the end, £5.05 a month for unlimited storage. Obviously it helps if like me you have a 90 meg down 20 meg up connection. But in the end you don't have to worry too much about storage, you know the data is all held online on spinning rust somewhere in the country, and if the vendor is worth their salt they're not going to lose your data. It also seems to hold a somewhat infinite file version history, which is pretty cool, should I ever need it.

I suppose its one for the future.

 

[Hugh Jarse]

Are these files now only on optical disc with no other copies available? If so, they are not backed up, no matter what format the disc is. If these are the only copy you have, then they are archived, not backed up.

Back up means you have at least one copy somewhere else.

I once lost work that cost me at least £2000 a few years ago. I decided to invest in a decent back up system after that. The problem is that until this happens to you, back up never seems like a good investment, as you see no benefit... and hopefully, you never WILL see the benefit. People like to spend money on shiny things.. not something they never see. It's like insurance... no one likes buying that either.

Here's my regime:
I back up everything to a second internal drive (automated).
That is then backed up to my main RAID6 server (automated)
That server is then mirrored to another RIAD5 server in my garage (automated).

At any one time I have 4 copies of all files, two of those copies are on systems with redundancy. The main server can actually stand t have two consecutive disk failures and survive. In order to lose data the main storage drive in my main machine would have to die, then my first back up drive, then TWO drives in my main server, then another drive in my mirror. I'd need FOUR disk failures simultaneously... and even then I'd probably be able to recover most of it.

It sounds extravagant, but all in I probably spend less than £1200 on the system.

When drives are full I either replace or expand. When I first built the system it used 1.5TB drives to give 3TB of space.... it's since been expanded using 4TB drives to give me 8TB of space. Hopefully when space becomes an issue again, there will be bigger disks available... if not, then I'll have to invest in a redundant archiving system.

If you earn money with your data, then you have no choice but to invest in something that makes it secure. To lose data now is very unlikely, and would probaly mean something serious enough to take out the whole house, and garage (which is 15 metres away).... in which case I've got much bugger issues than data loss!

Sorry... I'm not preaching... just serious about back up.

Been burned before.

The Blu Rays are stored at a storage unit I rent.

I usd DAT taps backup for a my business and home stuff. Tapes sre in a fireproof safe at the storage depot too.

Additionally I have ReadyNAS with 2x 2Tb HDD in RAID as a "way station" which backs up to DAT every 2 hours.

Last back up failure cost my business £8K and time worth twice that amount.

Over the top? Perhaps but once bitten twice shy.

S

 

[hollis_f]

Have you had a look at the cloud backup solutions?

The cloud is a good idea for storing data that you want to be able to access anytime, anywhere. As a backup solution it leaves a lot to be desired. Questions one should ask are...

• How long will it take to upload my 600GB of data?
• How long will it take to restore my 600GB of data, bearing in mind that my ISP is likely to have a limit on the amount I can download each month.
• What happens when my cloud storage provider goes bust?
• Who has access to my data?
• How is access to my data controlled?
• Which independent auditors have checked the security procedures of my cloud storage provider?

 

[Rojibo]

The cloud is a good idea for storing data that you want to be able to access anytime, anywhere. As a backup solution it leaves a lot to be desired. Questions one should ask are...

• How long will it take to upload my 600GB of data?
• How long will it take to restore my 600GB of data, bearing in mind that my ISP is likely to have a limit on the amount I can download each month.
• What happens when my cloud storage provider goes bust?
• Who has access to my data?
• How is access to my data controlled?
• Which independent auditors have checked the security procedures of my cloud storage provider?

The internet connection is going to be the main limiting factor, in my case it would take 18 hours to download 600GB, which for my personal data I deem a suitable RTO. My data is encrypted with a private key and any cloud provider worth their salt is going to be conforming to the revelant ISO standards, which you can easily check.

For me, it comes down to, would I like to store my data on the tin I use/administer/deploy daily? Yes.
Can I afford to? No.
Whats the next best thing? Oh I'll just rent a slice from a company.

Of course taking into account internet speeds YMMV, I just believe its the easiest way to get the same level of enterprise protection, not offered by the pc world type NAS devices.

 

[Pookeyhead]

Also not preaching or wanting to be the cause of additional outlay on hardware but ... what happens in the event of a fire?

I have the mirror in the garage... the garage is a separate building quite well separated from the house. Highly unlikely a house fire would damage the mirror's data, or vice versa.

The cloud is a good idea for storing data that you want to be able to access anytime, anywhere. As a backup solution it leaves a lot to be desired.

Agreed. Great for storage, but as back up, it really, really sucks. Far too slow. Even my full back ups over gigabit ethernet can take 40 minutes (my system does a full back every 7th incremental) and that's at a flat out 112MBytes/sec.. even a relatively fast broadband connection.. let's say 50Mbit will only top out at just under 6MBytes per second... and that's assuming you have no other overheads at the time.... and that you have synchronous upload/download speeds... which most do not.

 

[neil_1980]

Doesn't help you right now but I don't think my back up is too bad for redundancy if you're looking to reviews the way you do it for the future.

MacBook backs up to my readynas duo (2x2TB drives in raid -both drives different manufacturers so likely hood of failing at the same time is slim).

The readynas can backup over USB so have a 2TB drive that I backup the NAS too over USB and then take that drive off site.

This way my data is on 4 hard drives at 2 locations and hasn't cost too much.

Also when I've got a new set of photos I upload the ones I want to keep to Dropbox... That way they're off site too until I take the offsite drive home to back up.

Only weak point that I can see is if there was a fire whilst my offsite drive was home backing up but an extra drive at work and rotating which I take home would sort that.

I dare say there are better ways to do this but based on time and cost and simplicity it's hard to beat the above.

 

[Rojibo]

Agreed. Great for storage, but as back up, it really, really sucks. .

I completely disagree, perhaps I should have used the term cloud backup, rather than cloud storage. If you don't live out in the sticks the time to perform incrementals is short to none, and you're backing up at a block level. Sure the initial seed takes time, but once you're past that point you're laughing.

I'm able to attain a 5 minute RPO thanks to the magic of asynchronous transfers which I cannot do with a cheap NAS box and faffing around with tapes.

But then again, I suppose I don't have an online disk to disk backup estate thats >1.5 PB in size... oh, wait.

 

[Almost_Invisible]

I have the mirror in the garage... the garage is a separate building quite well separated from the house. Highly unlikely a house fire would damage the mirror's data, or vice versa.

That solves that problem. Having a house the size of a garage I tend to overlook the possibility of seperate buildings

 

[Pookeyhead]

I completely disagree, perhaps I should have used the term cloud backup, rather than cloud storage. If you don't live out in the sticks the time to perform incrementals is short to none, and you're backing up at a block level. Sure the initial seed takes time, but once you're past that point you're laughing.

I'm able to attain a 5 minute RPO thanks to the magic of asynchronous transfers which I cannot do with a cheap NAS box and faffing around with tapes.

But then again, I suppose I don't have an online disk to disk backup estate thats >1.5 PB in size... oh, wait.

Good luck when you need to back up your entire system, not just your images. I'm not sure if your service allows that absolute system level restoration, right down to the last saved position of every icon... but if it does, one day you might need to restore, and that could be terrabytes you end up having to download.

I could format my drives, and restore back to how I was in less than an hour. I reckon you'd be dead in the water for over a day... possibly longer. No idea if you rely on your computers for work or not, but waiting for hundreds of gig's, or possibly several terrabytes of data to download doesn't seem like a great idea to me. Then there's the issue of whether your ISP has truly unlimited download limits or not... many don't.

If it all goes fubar, I want to be back up and running ASAP.

 

[Rojibo]

If my os drive failed, the solution is pretty simple, re-install the OS on a new drive. Back up and running in probably less than an hour.

If one of my data drives failed the solution is pretty simple too, replace it and pull everything down from across the WAN because I'm lucky enough to have a massive pipe I can happily abuse.

At the end of the day I much rather trust enterprise class equipment with my data than some shoddy little home nas box.

Still living in the dark ages with the mentality against IAAS/Hosted solutions, how would you recover from a flood, for example? A proper disaster.

 

[Pookeyhead]

At the end of the day I much rather trust enterprise class equipment with my data than some shoddy little home nas box.

I don't use a shoddy little home NAS box

 

[steveo_mcg]

Still living in the dark ages with the mentality against IAAS/Hosted solutions, how would you recover from a flood, for example? A proper disaster.

Simple, weekly backups to the office means even if my house burns down most of my stuff is safe and in the event of a flood affecting my office we'd all have more serious issues ;like why the sea was now 100m higher than it should be..

 

[Rojibo]

I don't use a shoddy little home NAS box

So you're using a NetApp device then? Or did you go with EMC VNX?

 

[Pookeyhead]

Neither.

 

[DuncanF]

I have to agree that the cloud is a great form of backup. I use MS SkyDrive for my photos. I have a NAS running Windows Server 2012 and the share for images is part of the SkyDrive footprint. So if I edit an image on my desktop, once saved it is in the cloud a few seconds later. Added to that I can have synchronous local copies on my numerous devices using SykDrive. So I can save/edit on a laptop when on the road, and a few seconds after it is synced to the cloud it has also found its' way onto my NAS. Copies of images everywhere ...

I'm really not concerned about hardware dying. Desktop and laptops abound. What I am concerned about is having my data on a resilient infrastructure. To recreate that I'd have to augment my NAS with offsite backups. I'm happy to let MS take the strain.

I have fast fibre broadband so the initial upload took around 24 hours. It would have taken three to four weeks on my old copper broadband, so my solution may not be viable for everybody.

 

[Pookeyhead]

Sky Drive.. 7GB.. great LOL. I've got 3TB of images alone. Then I've got the rest of my stuff. Then there's the ability to get back up and running should my system drive crash. It's all very well saying you can re-install windows in an hour.. yes, you can, but then you have to re-install all your apps and get your system as you like it. Not everyone uses windows straight out of the box.. some of us have it heavily customised.

What no cloud system has allowed me to do so far, is be able to boot from a recovery disk, and simply put an image of my whole system back as it was in less than 30 minutes. If I Was to switch from my own servers to a cloud system I'd lose that ability. Also, if I Was to switch from my servers to a cloud system my initial back up, including my wife's system and work would be around 23TB... even on fast fibre, I'm not waiting for that.

Skydrive is useful for remote syncing files, but it sucks as back up.

 

[DuncanF]

As I said not for everybody. I actually have 125gb of space on Skydrive for images. That works for me.

My computers are separately backed up to the NAS using Acronis. So I could restore as needed very quickly. But it is all moot. I have a desktop and two laptops. The software I need is on all of them running in VMs, also backed up to the NAS . So they are just tools to allow me to access data. My RTO is therefore 0 (pick up a laptop and boot a VM) and with a sync to SkyDrive my RPO is as close to 0 as I can get without having some sort of synchronous mirror.

You seem very defensive about this ("Sydrive ... sucks"), but as I say each to their own. Your RTO of 30 minutes suits your needs.

 

[Rojibo]

I wouldn't worry about it too much Duncan mate, some people are too stuck on thier ways to think about doing things differently. Chances are if you need to have copies of your system images like that, your OS is installed poorly, and your using low end tin.

The cloud is a fantastic backup solution when you've got the bandwidth. I've just checked our internal cloud solution and we're backing up 2PB (that's 2,097,152GB), more than matey boy could ever dream of.

When you're storing everyhing under the sun, for home backups, you're not doing it cleverly.

 

[a1ex2001]

It's a shame this thread has gone a bit off track and turned into a bit of a bitching session as there was some good information and ideas being posted that people might have found useful.

For me the key thing isn't how you do it but that you think about it and do it! Some people will prefer tapes and DVD's others will be happy with an external hard disk the end result is the same. So everyone remember unless you have multiple copies in different physical locations you stand to loose everything in an instant.

 

[Almost_Invisible]

When you're storing everyhing under the sun, for home backups, you're not doing it cleverly.

That has me puzzled. How can a backup not contain everything?
Unless you mean copying everything every time rather than only changes, but even then sometimes imaging a drive takes very little additional time over synching backups.

 

[JonathanRyan]

Wow. People get pretty angry over backups.

Just a heads up, Time Machine is awesome until it breaks. Then it really really REALLY sucks.

Every backup ever is basically stored in one file. If that corrupts then the only way to start running backups again appears to be to delete that file (which actually isn't that easy). Which deletes all backups ever.

Ask me how I know.....

 

[Janesy B]

2TB USB3.0 WD Passport and I use free file sync, I've never had any issues with data loss when restoring.

 

[Rojibo]

That has me puzzled. How can a backup not contain everything?
Unless you mean copying everything every time rather than only changes, but even then sometimes imaging a drive takes very little additional time over synching backups.

Well everything that you want to keep following your Hard disk dying. Theres no point in keeping those GB's of Log files, temporary installer files, contents of the recycle bin, temporary internet files, I even knew some guy who insisted on keeping a copy of his page file, and to be honest I'm not sure how he managed that one.

The point is when a HDD dies are you going to want back all 600GB of Photos straight away? Probably not; you just need the last set you were working on to get to the client, so that 600GB becomes 10GB and you get up and working on that straight away rather than let a job restore the entire contents. It's just more efficient.

Oh and by the way if you want to do block level incrementals to an external storage device, or a different PC, the client from Crashplan is really good, and free.

 

[Pookeyhead]

I wouldn't worry about it too much Duncan mate, some people are too stuck on thier ways to think about doing things differently.

Of course.. that's why... nothing to do with the fact that it would take about a week to upload all my crap to the cloud :bang:

The cloud is a fantastic backup solution when you've got the bandwidth. I've just checked our internal cloud solution and we're backing up 2PB (that's 2,097,152GB), more than matey boy could ever dream of.

And for most of us on DSL that's perfectly normal of course . I've just worked out... at my 20Mbit speeds, it would take 9 days to initiate a back up set. The cloud may be brilliant with a hugely fast connection, but hardly anyone has access to the speeds you refer to, so leave your e-penis out of the thread and get real. For most people with average upload speeds, the cloud is a pain in the ass, and a well sorted home server with redundancy is a perfectly acceptable solution.

 

[Rojibo]

For most people with average upload speeds, the cloud is a pain in the ass, and a well sorted home server with redundancy is a perfectly acceptable solution.

I feel its far less than you think, given how fast fibre has spread across the country. I'm also not saying that a DIY home server isn't a useful solution, it is given the correct use case (and if its built properly, please for the love of god don't use software RAID).

However, I am refuting the point you're making by calling a WAN backup solution useless. Because for 90% of home users, it's probably the most cost effective and efficient method to cover themselves in the event of a disaster. It should not simply be dismissed because it does not fit your use case.

 

[Janesy B]

Of course.. that's why... nothing to do with the fact that it would take about a week to upload all my crap to the cloud :bang:

Realistically, once everything is up there - how long would an incremental backup take for you?

I think it's worth the initial pain even if you store the most important files to you off-site and on the cloud. FTTC is fast spreading and I've just gone from 0.8Mbps upload to 18mbps, as long as you're not out in the sticks you will probably find yourself in the BT/Plusnet fibre coverage area, if not you will be soon.