Digital paranoia anyone ???

I use a Netgear Readynas that provides 4 x 1gb drives, which gives 3gb of fault-tolerant storage. Any one disk can fail and you lose no data. It is fed by a UPS which shuts down gracefully in the event of a power outage.

It's really not that expensive compared with the value of the data.

But that should not be the only copy. How far you go depends on a lot of factors, but nobody ever regretted having too many backups :)

Graham
 
DiddyDave said:
Ahead of you there Steve :)

The 3 units will be used in rotation so each one goes off-site for a month before coming back for two

Fingers Xd that should do it

DD
Click to expand...
I think you've got it pretty much sorted. It's easy to become paranoid about having endless back-ups, but you have to draw the line somewhere.

I have loads of photos from the '80s of things that I will never be able to recreate, and I have the prints stored in my back bedroom, and the negatives in a drawer just above them. I am more paranoid about those than the ones I have stored on 2 hard drives.

My long term plan is to get around to scanning most of them, but that will take a long long time, and I don't think the other half would appreciate all the time it will take!
 
Goody13 said:
I have the prints stored in my back bedroom, and the negatives in a drawer just above them. I am more paranoid about those than the ones I have stored on 2 hard drives.
Click to expand...

Why the paranoa?

Goody13 said:
My long term plan is to get around to scanning most of them
Click to expand...

I personally think it's a waste of time scanning them just for the sake of having copies. Scan them when you want to do something with them e.g. making a print.

A scan can never be considered as an archive copy and it is very probable that the negatives will continue to exist long after your scan files have disappeared.


Steve.
 
My paranoia is only a relative thing. What I should probably have said is that only having one set of negatives and prints is of more concern than having two electronic back-ups of my more recent photos.

The main purpose for me scanning them is to get them on to my new Fotopic site for people to see, plus I am also more likely to look at them if they are in electronic form than I am to get out old packets of photos/albums!
 
Hi David101, unfortunately you are not correct there. Raid 0+1, you /can/ loose two drives and not loose data, but in reality, because of the way the extra load is put on the second disk in the pair, you will more often loose data after the second disk failure.

DiddyDave. There is nothing wrong with taking lots and lots of backups. You can never be too safe. At the moment, I am considering having my main data on a raid 5 system, as my main working system. Then I will take a backup to an external disk drive. Once a month, I am going to clone that external drive to an offsite area (well, in fact I am going to retrieve the offsite disk drive, and swap it with the current external one). This need not be expensive, consider a family member or a friend who doesn't mind you popping in once a month. The reasoning behind this is if you are burgled or the premises are burnt down. Also, each month, I can check the consistency of the offsite backup

On the raid side of things, a quick explanation. These explanations are mainly based on /hardware/ raid controllers. Software raid controllers may not offer all of the benefits (hardware raid is a separate device inside the computer, software raid the operating system performs the tasks). Also assumed that all disks are the same size. With hardware raid based systems, you can usually take disks out and replace them whilst the machine is still in use. (not with a raid 0 of course)

Raid 0. This offers no redundancy, but allows you to read and write data faster. Data is interlaced between disks, You can have multiple disks in the raid set (2+). The speed benefit is gained by having twice as much cache local to the disks. Cache being a buffer memory based on the disk controller itself. Often called stripe raid. Storage size is about the size of all the disks added together.

Raid 1. Mirror. Data is sent to both disks at the same time, in a mirror. If one disk fails, the other is used immediately. Often raid controllers will continually/schedule checks to ensure that the data on the disks match, and alert you if there are any errors likely to be encountered (not an awful lot it can do to fix them though). Mirrors have to be in pairs.

Raid 0+1. This is a stripe of mirrors. This gives you the resiliency of a mirror, you can loose a disk, but with the extra speed performance of a stripe. However, experience shows that should one disk fail, the other disk in its mirror, starts to get a lot more use. This can cause this disk to fail quicker than the other two disks left, meaning that you will more often than not, loose data on a second disk failure. For raid 0+1, you have half the disk space, minimum requirement of 4 disks, any even number above that is valid.

Raid 3. This raid system is no longer used (in preference to raid 5 if it exists), it is however easier to explain raid 3 then raid 5. With raid 3, the computer writes the data to all of the disks less 1. So if you have 4 disks in a raid 3, the data would be written to disk 1, 2, 3 . For disk 4, a checksum of the data is written. This checksum allows the computer to calculate whether any of the items, 1, 2, 3 have changed, and if they have, it can re-create the value that they had (conversely, of the data on disk 4 has changed, it can recreate that). With a raid 3, if one disk fails, then there is no loss of data. You have to have a minimum of 3 disks. You loose one disk space for the parity information.
The downside found with raid 3, was that when a disk fails, the amount of extra usage put on the disk for the parity calculations, caused it to fail quicker.

Raid 5. This is a similar theory to raid 3. The difference is that instead of there being 1 disk reserved for parity, the parity is interspersed with the standard data. Similar to the below (A,B.C being the data, P being the parity).
ABCP
BCPA
CPAB
PABC

Most (proper) hardware raid systems will also perform consistency checks of the data, and should there be a problem showing with the disk, they will copy the failing disk to a spare disk within the system, usually called a hot spare. It isn't infallible, but around 80% of the returns I see are from systems which have correctly determined an upcoming failure and automatically copied and replaced the failing disks..
 
I've just been looking at BD-R's again as their archival life is supposedly several times longer than DVD-R. Last time I looked it just wasn't cost effective but my first quick search turned up 10 BD-R's for £17.99 which works out at about 7p per GB. A 1TB external HD is around 6p per GB now so, aside from the initial purchase price of the BD drive, they're pretty much equal.

So is it now, at last, worth considering spending a couple of hundred quid on an external Blu-Ray drive?
 
Goody13 said:
My paranoia is only a relative thing. What I should probably have said is that only having one set of negatives and prints is of more concern than having two electronic back-ups of my more recent photos.!
Click to expand...

I don't think you need to worry. Negatives only really become un-useable after a catastrophe such as a fire or a flood. Digital files become corrupt on a normal day to day basis just by the nature of the storage medium not being perfect.

We have negatives and prints from my parents, grandparents and great grandparents which are fine but I have lost files from the last five years due to a hard drive failure.

The three drive rotation method DiddyDave suggests seems to me to be a very good one - especially for a photography business. For personal amateur use I'm sure it's fine too but will anyone carry on the regime once you are gone?


Steve.
 
Steve Smith said:
I don't think you need to worry. Negatives only really become un-useable after a catastrophe such as a fire or a flood. Digital files become corrupt on a normal day to day basis just by the nature of the storage medium not being perfect.

We have negatives and prints from my parents, grandparents and great grandparents which are fine but I have lost files from the last five years due to a hard drive failure.

The three drive rotation method DiddyDave suggests seems to me to be a very good one - especially for a photography business. For personal amateur use I'm sure it's fine too but will anyone carry on the regime once you are gone?


Steve.
Click to expand...
Highly unlikely, but as my photos are mainly for personal enjoyment, they will not be of much interest to others. On the other hand, the ones from the '80s of things long since gone may be of interest to others.

It was only fire or an escape of water from above that is of any remote concern to me as far as the negs are concerned as I have always stored them in their original packets at room temperature, so feel I have taken pretty good care of them over the years.
 
Why not have an online back-up as well? That way if there's a fire or a burglary and all your kit is lost, you can just down load all the files again.

I use carbonite but that doesn't backup from external HD's. When the subscription is up I'll change to the other one (mozy??) which does.
 
Goody13 said:
but as my photos are mainly for personal enjoyment, they will not be of much interest to others.
Click to expand...

Don't be too sure. Most of the old photos we look at and are of interest to us now probably held very little interest to anyone else in their day. The passage of time makes even the most mundane of images interesting.


Steve.
 
I run a proliant server with freenas holding a backup of everything with my macbookpro and mac pro mirroring their photo directories and a further backup on ext HD. BUT the important thing is that I also then backup the lot to another ext HD which I keep at work and exchange about once a fortnight or so dependant on what I am doing. You have to think WORST CASE and that is unfortunately a fire or large burglary. It is all well and good a having multiple copies of the same stuff in one place but you could loose them all. A proper fire safe is a possible too.
 
cowasaki said:
BUT the important thing is that I also then backup the lot to another ext HD which I keep at work.
Click to expand...

How do you confirm that the files you are backing up are good? Is it possible that you are making perfect copies of corrupt files?


Steve.
 
Oh and I was looking at an online system. SKY do one but it is PC only for the automatic backup software which is a pain and the actual storage is 60Gb for £4.50 per month. This would be big enough now but I would already fill it!
 
Steve Smith said:
How do you confirm that the files you are backing up are good? Is it possible that you are making perfect copies of corrupt files?


Steve.
Click to expand...

The mirroring software does that so quite happy. But just looking now at something someone pointed me at a while ago but didn't look at.

MOZY - unlimited space backup at $4.95 per month :) Just looking at that and the pro version now as they are both Mac compatible.
 
Well I've just signed up to MozyHome and it looks pretty good! Fully automated backups to a secure server in the background!

If anyone does decide to join it though (because of this post) then please can you enter the referral code 9EZJV6 in the box so I can claim a referral and offset my storage costs.
 
a triple backup contigency should be fine....airliners usually have 3 sets of control lines, for instance (not counting the sioux city crash :-p)
 
I don't back mine up at all.:shrug:
 
cowasaki said:
Well I've just signed up to MozyHome and it looks pretty good! Fully automated backups to a secure server in the background!

If anyone does decide to join it though (because of this post) then please can you enter the referral code 9EZJV6 in the box so I can claim a referral and offset my storage costs.
Click to expand...

Mozyhome is awesome- only downside is getting the data up there!
 
blue max said:
It's really not that expensive compared with the value of the data.
Click to expand...
Indeed. Any business relying on digital media is mistaken if they think they do not need a disaster recovery system in place - but LOTS do not have systems that in any way robust!

Off-site storage has to be the 'best' solution - banks and the like tend to back-up to data stores on other continents - but whatever you do make sure that you test your method from time-to-time and not find that it hasn't worked when you are in a blind panic when you eventually need it!

As to the future (for everyone) - beware the changes in hardware. They sneak up on you and you'll find too late that you can't read the data you have saved. I knew someone who hoped to get some old data off some old 5¼-inch floppys but had no way of reading them. They had an old floppy drive but could not find any connector on any modern computer motherboard for the ribbon cable...... When they did they could not copy the data off onto a USB stick as the computer didn't have a socket (and only had an coax network card!). I have some old SCSI drives here and no SCSI interface to connect to and read them. Can't remember if they are SCSI1, Fast SCSI, Fast/Wide SCSI, Ultra SCSI, Ultra wide SCSI, Ultra SCSI2........ or (without digging them out of a box) whether they are 50 or 68 pin. Then will the software support the hardware.... and so on.

My solution - print the pictures you want and hang them on the wall or put them in an album. In reality you'll get bored with them or run out of space before they are 'lost'.
 
coldpenguin said:
No, please before using raid 10, 30 or 50, please understand that this only really gives you single disk redundancy.
If you want more than single disk redundancy and stripe, you need to be at least raid 60
Click to expand...
umm, if you're a **** IT sysadmin and dont actively replace faulty hardware then you'll have a problem with RAID10...

"All but one drive from each RAID 1 set could fail without damaging the data. However, if the failed drive is not replaced, the single working hard drive in the set then becomes a single point of failure for the entire array. If that single hard drive then fails, all data stored in the entire array is lost. As is the case with RAID 0+1, if a failed drive is not replaced in a RAID 10 configuration then a single uncorrectable media error occurring on the mirrored hard drive would result in data loss. Some RAID 10 vendors address this problem by supporting a "hot spare" drive, which automatically replaces and rebuilds a failed drive in the array."

you're very unlikely to get a harddisc failure on 2 discs in the RAID1 set - the chance of that are very minute unless you let someone run through your datacentre with an electromagnet. if a backplane or controller in your SAN enclosure goes, then that's not destroying data on the disc.

edit: in the 10 years i've been working for the company i work for, i've only ever seen 2 harddisc failures on a server level. 1 was because someone dropped the actual server while moving it (1 of the discs in a RAID1 set was lost) and the other one was a general failure in a RAID10 set on a SAN.

no data was lost in either case. but i agree, you should have an approved DR plan in place, which includes offsite backups etc.
 
The concern I have with all these online backup firms is that you are dependent on them surviving. Lots of start up companies have a habit of disappearing and some of the cheap ones can't be making money.

The whole "we encrypt your data" to me means "we can charge you more to get it back later if you need it and we change our policy" or "if we go bust, its definitely lost"

Currently, I'm sticking with RAID5 NAS and then external drives stored in a Peli case in my car boot...
 
grumpybadger said:
The concern I have with all these online backup firms is that you are dependent on them surviving. Lots of start up companies have a habit of disappearing and some of the cheap ones can't be making money.

.
Click to expand...

But the chances of THAT happening AND your HD going **** up at the same time must be pretty remote......

For me the only drawback is remembering to leave your computer switched on for long enough for the files to upload.
 
jerry12953 said:
But the chances of THAT happening AND your HD going **** up at the same time must be pretty remote......

For me the only drawback is remembering to leave your computer switched on for long enough for the files to upload.
Click to expand...

But there is no point having a backup you can't rely on. I reckon it is less reliable than a hard drive and wonder if people are kidding themselves. Anyway, if you're happy that's fine. Its just not for me.

That and the fact it would take over 250 days 24/7 to upload my data I need backing up!
 
recrudesce said:
umm, if you're a **** IT sysadmin and dont actively replace faulty hardware then you'll have a problem with RAID10...

-----
cut the GOOD explanation (and I mean that in a non-sarcastic way), but this is the bit most people don't read.
-----

you're very unlikely to get a harddisc failure on 2 discs in the RAID1 set - the chance of that are very minute
Click to expand...

I have worked for an OEM for too many years now. The first item you listed is very common. Even if the machine is beeping and flashing a red light, you often get an admin who will walk in, and press the ignore button. I have worked with re-selling Dell and HP equipment a lot as well. Often it can take 7+ days to get replacement disks from them. However, in anything more than a raid1 mirror, I always ensure that there is a hot-spare (and I advise the customer to take backups as well, but they rarely listen). I once had a customer with a 9 disk raid 5+hot spare. They complained when they lost data. I recovered the machine, looked at the logs, and asked why over the last 4 months they had ignored the beeping, the flashing lights, and the emails that the system had been sending to them, after 1 disk failed, 2 disks failed, then the third and final redundant disk gave up.

Unfortunately, my experience of raid 1 mirrors, and especially striped mirrors, is that the second disk in the mirror gets a pounding as soon as the first one has failed. Often the disks are from the same batch, and same age, and tend to not put up too well to the pounding. However, most of the places where I have put raid 10 has been into database stores etc., a slightly different workflow to a jpeg server.

The advantage to mirrors over raid 5 though, is that when the system comes back to me for recovery, it is often easier to reconstruct the data from a mirror than it is from a raid 5.
 
I have all my photos on a 1tb HDD. No redundency.

I really really really should think about backing them up. erp.... Optical media just doesn't seem like an option, even BluRays, they're expensive and slow to read/write. Considering 1000gb is <£50 now. Perhaps external HDDs is my way forward....
 
davidsteward said:
I have all my photos on a 1tb HDD. No redundency.

I really really really should think about backing them up. erp.... Optical media just doesn't seem like an option, even BluRays, they're expensive and slow to read/write. Considering 1000gb is <£50 now. Perhaps external HDDs is my way forward....
Click to expand...

That seems to me the best way. It's current and reliable. If it becomes obsolete, so does your computer.

You have to accept that there is a cost, a decision and a strategy involved. It will give you a warm glow to think that it is safe.

And really £70ish quid for a terrabyte is crazy cheap for a first step.

Graham
 
coldpenguin said:
I have worked for an OEM for too many years now. The first item you listed is very common. Even if the machine is beeping and flashing a red light, you often get an admin who will walk in, and press the ignore button. I have worked with re-selling Dell and HP equipment a lot as well. Often it can take 7+ days to get replacement disks from them. However, in anything more than a raid1 mirror, I always ensure that there is a hot-spare (and I advise the customer to take backups as well, but they rarely listen). I once had a customer with a 9 disk raid 5+hot spare. They complained when they lost data. I recovered the machine, looked at the logs, and asked why over the last 4 months they had ignored the beeping, the flashing lights, and the emails that the system had been sending to them, after 1 disk failed, 2 disks failed, then the third and final redundant disk gave up.

Unfortunately, my experience of raid 1 mirrors, and especially striped mirrors, is that the second disk in the mirror gets a pounding as soon as the first one has failed. Often the disks are from the same batch, and same age, and tend to not put up too well to the pounding. However, most of the places where I have put raid 10 has been into database stores etc., a slightly different workflow to a jpeg server.

The advantage to mirrors over raid 5 though, is that when the system comes back to me for recovery, it is often easier to reconstruct the data from a mirror than it is from a raid 5.
Click to expand...

Should us normal folks be able to understand this......?;)
 
jerry12953 said:
Should us normal folks be able to understand this......?;)
Click to expand...

Sometimes my wife gets that glazed look in the eyes and nodding. So I change the subject to pink elephants hiding in the strawberry patch.
I can usually continue in this vain for several minutes

Anyway, I am normal
:)
 
You must log in or register to reply here.
Back
Top