Long term RAW file storage

wyx087

wyx087

Messages
3,175
Edit My Images
Yes
How do you store your family photo RAW files? (not professional/work shots)
What is your storage and backup regime for those RAW files?

I'd like to keep the RAW files long term, with backup. They may never get looked at again, so I don't need it to be as accessible as on an always-on NAS. My NAS is 90% full, so instead of buying more storage, I think I need to start using appropriate storage method for data according to how often and quick I need them.

As with all my data, there has to be at least 2 copies of it somewhere. My main NAS is mirror backed up to another NAS that only wakes up once a week to receive backup. But on the other hand, offline data like these should be fine with 1 copy as long as the storage media is solid.

Possible solution?
- Portable HDD, managing it gets messy if I want multiple backups. How reliable are HDD's if kept in a cupboard for 10+ years?
- Amazon glacier, not a fan of having large on-going fee. Only using it as off-site storage for my picked photos in case of house fire.
- Burn to DVD's? waaaaay to much effort and time consuming, I could have 50GB of data in one Lightroom catalogue export.
 
2 Time Machine Discs - one kept in my desk drawer, the other in the not-attached-to-the-house garage. Both backed up everytime I switch on the iMac. Another disc drive with RAW files in the garage.

Finished TIFFS and RAW files uploaded to Amazon Photos, (Prime member so included in Prime sub.)
 
I keep Raw files and final edited files (TFF/PSD) and from 2010 onwards are all on my current data disk. They are also backed up on a NAS and an external USB HDD which is normally kept in a fire resistant safe. Earlier archived Raw and final edited files are also backed up on the same place. I only have two LR catalogues the current and archived and each has about 30,000 images. Total storage required for both sets of images is almost 4 T Bytes. I do not normally keep JPEG files permanently as they can be created from the TIFF/PSD files as needed. I do have a handful of my best images stored in free cloud storage. It is probably pointless using DVD's as they are not permanent as I discovered to my cost; they may be unusable by around ten years. In backing up I do not always just add changes but from time to time make a fresh back up which takes many hours. It is possible that data on an HDD could start to deteriorate after many years so rewriting from time to time (say every 2 years) should prevent that.

Dave
 
You can get writable Blu-ray disks up to 128Gb capacity now, but I'm not sure what their archival longevity is like.
 
Key is multiple backups in different locations as that mitigates against problems including online options.

No technology is fool proof - mechanical problems can hit spinning platter hard drives. Flash drives could give problems. Online storage providers are subject to going bust, being take over, hacked. Family photos may get referred to in 30+ years time, so backup plans have to factor in extracting information from old technology.
 
I'm trying to find a storage solution that's not too expensive to store TB's of data and can last a long time, at least my lifetime if possible.

I think keeping it on NAS or Timemachine drives is not a good solution because it's not cost effective and we don't need 24/7 access to it so it should not be 24/7 connected, liable to be hacked or infected by encryption-malware.

andrewc said:
Family photos may get referred to in 30+ years time, so backup plans have to factor in extracting information from old technology.
Click to expand...
Exactly! Perhaps being Lightroom catalogue export, also put Lightroom install and Windows image with it?

So what do you suggest? ;)
 
All this sounds incredibly anal to me, and far too hardware-reliant. What about an album or two of prints? Then we could all just relax.
 
I think the one safe bet here is that a Windows image taken today will not run on then current hardware in 30+ years time. I've seen that happen already with modern hardware and XP images. Unless you want to be restoring (optionally updating) and resaving backups every few years as standards evolve, the only long term future-proof solution is something human readable that can be re-digitised at some point in the future, should that be required.
 
The JPEG is almost 30 years old and I have a fair number of 20+ year old JPEGs that are still readable. The CDRs were burned on a Win95 machine IIRC.
 
Yes, to be fair I'm not talking about the those photos that have been picked and processed. I have them in JPEG and backed up in many places. The best ones are printed, although I've discovered a few days ago printed photo books will not survive toddler's fingers.


It's the RAW files, sometimes I see an old photo I think I could have editted differently, I'd like to have the option to go back and edit it. It's unlikely in 40 years time I'll go back to edit the same image, but it would still be nice to have this capability considering the price of a 4TB HDD these days.

The idea of the thread is more about asking how to store and backup all your old RAW files offline and preferably not need to tend to it every few years. Are portable HDD the way to go, for offline storage?
 
ANY storage is only temporary. Some media might last longer than others but there's a saying that there are two types of hard drives - those that have already failed and those that will fail. Same with solid state - even memory cards have been known to pack in.
 
wyx087 said:
How do you store your family photo RAW files? (not professional/work shots)
What is your storage and backup regime for those RAW files?

I'd like to keep the RAW files long term, with backup. They may never get looked at again, so I don't need it to be as accessible as on an always-on NAS. My NAS is 90% full, so instead of buying more storage, I think I need to start using appropriate storage method for data according to how often and quick I need them.

As with all my data, there has to be at least 2 copies of it somewhere. My main NAS is mirror backed up to another NAS that only wakes up once a week to receive backup. But on the other hand, offline data like these should be fine with 1 copy as long as the storage media is solid.

Possible solution?
- Portable HDD, managing it gets messy if I want multiple backups. How reliable are HDD's if kept in a cupboard for 10+ years?
- Amazon glacier, not a fan of having large on-going fee. Only using it as off-site storage for my picked photos in case of house fire.
- Burn to DVD's? waaaaay to much effort and time consuming, I could have 50GB of data in one Lightroom catalogue export.
Click to expand...


I think cloud based is the most sensible solution. Make the physical job of looking after it someone else's problem
 
In 30 years time there will be companies specialising in converting and extracting data from extinct kit, in the same way that people can get cine film and old video digitised, and I'm sure JPG's and RAWs will be able to be converted to whatever image format gets beams directly into the brain in 2050
4
 
Ive looked into this and there are some dvd's that are designed for long term storage as in decades. They will be much safer than hard drives for storage of the durations you are thinking of.
I would also store processes jpegs alongside the raws, and consider using DNG's instead of the raw file. It may be easier in decades time to open a dng than a proprietary raw file, though i would guess that raw converters will be available for most cameras for a long time to come.
 
wyx087 said:
How do you store your family photo RAW files? (not professional/work shots)
What is your storage and backup regime for those RAW files?

I'd like to keep the RAW files long term, with backup. They may never get looked at again, so I don't need it to be as accessible as on an always-on NAS. My NAS is 90% full, so instead of buying more storage, I think I need to start using appropriate storage method for data according to how often and quick I need them.

As with all my data, there has to be at least 2 copies of it somewhere. My main NAS is mirror backed up to another NAS that only wakes up once a week to receive backup. But on the other hand, offline data like these should be fine with 1 copy as long as the storage media is solid.

Possible solution?
- Portable HDD, managing it gets messy if I want multiple backups. How reliable are HDD's if kept in a cupboard for 10+ years?
- Amazon glacier, not a fan of having large on-going fee. Only using it as off-site storage for my picked photos in case of house fire.
- Burn to DVD's? waaaaay to much effort and time consuming, I could have 50GB of data in one Lightroom catalogue export.
Click to expand...

With amazon prime you get unlimited photo storage including raw files. I've only tested it with Canon, Fuji and Leica, it works with my Canon and Fuji files. That might be an option if you have prime, or were considering it
 
I think it depends on how much you want to back up. Most data saving methods have been superseded as tech evolves, and length data is supposedly 'safe' varies. Floppy Disk > CD > DVD > Blu Ray, HDD > SSD. Add to those, memory cards and USB drives. All have pro's and cons.

I have decided to go with HDD's for now. I have all my image on an internal HDD in the PC. An 8Tb Samsung HDD backs up the whole PC and Laptop memory, including all my images. And a separate portable 4Tb Lacie HDD that backs up just the images. I have older HDD's of various capacities which have a lot of images up to a certain point when they were full, or close to full.

For the amount of data I have, a cloud option is not feasible from a cost, and upload would be slow, and download speed to get a lot of data back, should the worst happen, would take awhile too. I can see a cloud option for the final keeper Jpeg and RAW file, but not for everything.
 
jimmyriddle said:
With amazon prime you get unlimited photo storage including raw files. I've only tested it with Canon, Fuji and Leica, it works with my Canon and Fuji files. That might be an option if you have prime, or were considering it
Click to expand...
It would be interesting to see what they think 'unlimited' is. I have a friend who was on an unlimited data contract with 02 about 6-7 years ago, which wasn't unlimited any more when he was approaching 3Gb a month. ;)
 
redhed17 said:
It would be interesting to see what they think 'unlimited' is. I have a friend who was on an unlimited data contract with 02 about 6-7 years ago, which wasn't unlimited any more when he was approaching 3Gb a month. ;)
Click to expand...
40,000 files up there from me at the moment
 
Rapscallion said:
Ive looked into this and there are some dvd's that are designed for long term storage as in decades. They will be much safer than hard drives for storage of the durations you are thinking of.
I would also store processes jpegs alongside the raws, and consider using DNG's instead of the raw file. It may be easier in decades time to open a dng than a proprietary raw file, though i would guess that raw converters will be available for most cameras for a long time to come.
Click to expand...
I don't think any manufacturers RAW files will become unusable unless they were not in business, and a very niche brand. It will not be in any company who produces editing software's interest to not include the option to read every file from every camera, especially when the code to do that has existed for possibly many years. Time will tell though.

I considered DNG in the past, but when Adobe Camera RAW and LightRoom where able to make lens adjustments as part of the RAW processing, it was beneficial to keep the original RAW file, because I don't think that option was there when converted to a DNG. That may have changed, I don't know.
 
Last edited:
The problem with online cloud storage is that its only secure as long as the bills are paid, and if for whatever reason that bill doesn't get paid, your data could easily get deleted. That could be due to illness, divorce or simply an oversight because someone forgets that photo storage is included with Amazon Prime and someone cancels the service.

A multi facetted approach is needed. Whilst DVDs only hold 4.7Gb, archival blue-rays hold 100Gb and supposed to last hundreds of years provided they are stored correctly. A pack of 5 is £ 65, so a 100Gb backup could be given to 5 family members to spread the risk, and should someone cancel Amazon by accident, copies are available.
 
andrewc said:
The problem with online cloud storage is that its only secure as long as the bills are paid, and if for whatever reason that bill doesn't get paid, your data could easily get deleted. That could be due to illness, divorce or simply an oversight because someone forgets that photo storage is included with Amazon Prime and someone cancels the service.

A multi facetted approach is needed. Whilst DVDs only hold 4.7Gb, archival blue-rays hold 100Gb and supposed to last hundreds of years provided they are stored correctly. A pack of 5 is £ 65, so a 100Gb backup could be given to 5 family members to spread the risk, and should someone cancel Amazon by accident, copies are available.
Click to expand...
And don't get rid of the Blu-ray player/recorder. ;) :LOL:
 
Good shout on converting to DNG. I never used it and just read up on this, it would remove the need to save and later import the whole LR catalogue and allow me to import single photos with previous edits already applied.
Alternatively, use XMP alongside RAW files. Either way, I need to move away from using LR export catalogue.


For cloud storage, I did a quick calculation for 4TB of data:
- Amazon glacier: $0.004 per GB per month => $16
- Amazon photos via Prime: flat rate £8 per month with other benefits
https://www.amazon.co.uk/gp/help/customer/display.html?nodeId=GGU2SU8Y22DZYRMQ
But as said, using cloud storage has different set of problems to maintaing own copies, mainly the on-going cost.

The 100GB bluerays are said to have "projected lifetime of several hundred years".... hum....
https://www.amazon.co.uk/Verbatim-98913-M-Disc-100GB-Surface/dp/B011PIJPOC?th=1
40 disks for 4TB is manageable.


I'm leaning towards HDD for now. Get two external HDD, then give one to my parents, one kept on-site. Once space has been freed from my NAS, continue to do the same as previously, repeat as required.

Hope with two HDD's kept in storage, one will still work after 10+ years.
 
You must log in or register to reply here.
Back
Top