Mac Network Storage of Pictures, Lightroom and Backups

  • Thread starter Deleted member 49549
  • Start date
D

Deleted member 49549

Guest
Hi all,

My photo library is now getting close to exceeding my hard drive capacity so I need to spec myself a solution. I currently have this years pictures on my local drive, all previous pictures on a portable hard drive, and both of these are backed up using time machine. I have decided that I need to move to network storage. But have not yet worked out what I need or the best workflow.

My current thoughts:-

For my photo storage(approaching 500gb currently):-
  • Buy a nas drive (synology 2 bay possibly) with a raid configuration to allow for drive failure.
For backup
  • Buy another nas drive (possibly another 2 bay synology but with larger drives?)and software(possibly CCC) to do incremental versioned backups of the photo storage drive. I will also back up my laptop to this drive.
Off site backup.
  • Create a 2 x mirrors of the back up drives (or the photo storage?), rotated off site.
Is there a simpler / cheaper way of doing this? It is going to hit the bank a little more than I was intending (2 x nas, 4 x hard drives, 2 x poretable drives). But I am looking for a long term, expandable solution, and want to make sure all bases are covered.

Regarding workflow, for those using lightroom and network drives, do you initially import to your local drive, cull ,edit then move to the NAS, or is the NAS quick enough to do the initial cull and edit?
 
Last edited by a moderator:
I've never found a NAS drive fast enough to comfortably work from.

However,

1. I have a pretty low tolerance for slow stuff when I'm editing. In the initial cull, it's rare that a picture will stay on my screen for a full second.

2. I believe some NAS drives are much quicker on a Mac system than others.

3. Mavericks is coming/has come and Apple like to bugger about with their networking with new operating systems. Ask me to tell you the hilarious story of when they basically stopped Buffalo drives working (it wasn't all their fault - I despise Buffalo for their response but I still have a useless NAS with no easy way to access the data). You may want to wait until people have some hard data on which boxes work well with Mavericks.
 
I always used to run my working files from a nas and didn't find it particularly slow, however it was a) gigabit cabled, b) it's was an expensive smb unit and c) I now run my working files on a ssd and then move them off to nas when finished.

That said, if you don't need the network functionality (I.e. Several devices accessing the data) it may be worth looking at a das instead.
 
neil_g said:
I always used to run my working files from a nas and didn't find it particularly slow, however it was a) gigabit cabled, b) it's was an expensive smb unit and c) I now run my working files on a ssd and then move them off to nas when finished.
Click to expand...

Is that with a Mac? I'm finding gigabit cabled NAS slow on a Mac - other people use different brands and find them faster. It seems Apple aren't as compatible as you'd hope.
 
That was Win7.

We do use Gigabit on Mac at work to AFP and I wouldn't say it's slow running huge Illustrator files from server. SMB is another kettle of fish though, that gets a bit tetchy on a Mac.

I would advise making sure any NAS purchased to work with Mac is capable of AFP.
 
neil_g said:
That was Win7.

We do use Gigabit on Mac at work to AFP and I wouldn't say it's slow running huge Illustrator files from server. SMB is another kettle of fish though, that gets a bit tetchy on a Mac.

I would advise making sure any NAS purchased to work with Mac is capable of AFP.
Click to expand...

That could well be my problem. But my QNAP is still more sluggish than I'd expect and that uses AFP. (I assume it uses AFP as I just barely understand this area - I issue an AFP mount to mount it)
 
JonathanRyan said:
I've never found a NAS drive fast enough to comfortably work from.

However,

1. I have a pretty low tolerance for slow stuff when I'm editing. In the initial cull, it's rare that a picture will stay on my screen for a full second.

2. I believe some NAS drives are much quicker on a Mac system than others.

3. Mavericks is coming/has come and Apple like to bugger about with their networking with new operating systems. Ask me to tell you the hilarious story of when they basically stopped Buffalo drives working (it wasn't all their fault - I despise Buffalo for their response but I still have a useless NAS with no easy way to access the data). You may want to wait until people have some hard data on which boxes work well with Mavericks.
Click to expand...

Hi Jonathan,

sometimes i also go through my intial cull very quickly, marking any poorly composed or oof shots as rejects. So i am inclined to initially import to local drive, then move once the initial culling / edits are done.

It will be a few weeks before i will be buing a nas, so hopefully any problems with mavericks will show up.
 
JonathanRyan said:
That could well be my problem. But my QNAP is still more sluggish than I'd expect and that uses AFP. (I assume it uses AFP as I just barely understand this area - I issue an AFP mount to mount it)
Click to expand...

QNAP is a pretty well respected brand, id expect it to do AFP as standard.

might be worth checking your ethernet adapter on the mac has detected the speed of the link correctly.
 
neil_g said:
I always used to run my working files from a nas and didn't find it particularly slow, however it was a) gigabit cabled, b) it's was an expensive smb unit and c) I now run my working files on a ssd and then move them off to nas when finished.

That said, if you don't need the network functionality (I.e. Several devices accessing the data) it may be worth looking at a das instead.
Click to expand...

Will some NAS drives allow connecting of a portable hard drive? If so i will point my WDTV to the nas with my media HD attached, rather than unplugging the drive every time i move the wdtv. I also intend for the solution to be relatively long term, so am more inclined to NAS in case i add another desktop / laptop in the future.
 
Last edited by a moderator:
Going back to my initial question, how does that set up look? Any advice?
 
Rapscallion said:
Will some NAS drives allow connecting of a portable hard drive? If so i will point my WDTV to the nas with my media HD attached, rather than unplugging the drive every time i move the wdtv. I also intend for the solution to be relatively long term, so am more inclined to NAS in case i add another desktop / laptop in the future.
Click to expand...

Yup, Synology normally do if thats the make you were looking at (obv check the specs). It just shows up as a separate share on the NAS.

Rapscallion said:
Going back to my initial question, how does that set up look? Any advice?
Click to expand...

yeah its pretty much what i used to do. i used to use sync toy (i spent the couple of quid for the pro version) to sync up two drives and used the synology built in backup to run backups to USB drive for off site.
 
One more question (!) - would you bother with mirrored raid for the backup drive?

Another idea i have had is locating the backup nas drive at the parents house, and backing up remotely over the internet (after making the initial backup locally). This would remove the need for separate off site drives.

A colleague has also recommended qnap, have you used them?
 
depends how much storage you need for your budget.. it also depends how quickly you take your offsite offsite.

if youre having all of those drives at one site id look at UPS also. you can have all the raid you want and itll all come crashing down with a power event.

if youre going to look at offsite backup via the internet, you need to consider upload and download speeds.

ive never used qnap but they measure up very similar to synology who i rate highly.
 
Last edited:
neil_g said:
QNAP is a pretty well respected brand, id expect it to do AFP as standard.
Click to expand...

It probably does though it's quite an old one (TS-109 - not the "pro") - one day when I'm not busy I'll spend some time and figure it out. The QNAP is basically an iTunes library and it seems noticeably quicker accessing via the Sonos system than from iTunes on the Mac.

Rapscallion said:
One more question (!) - would you bother with mirrored raid for the backup drive?
Click to expand...

I would (in fact I do). But you need to bear in mind that I make my living from pictures ;) I don't know your situation, but this seems quite a complex setup if it's for home use. You can end up spending a lot of money and time doing nothing more than backups when you really want to spend your time on enjoying photography.

One way to look at it is.......if your drive failed tomorrow, would you pay me £500 if I could guarantee to recover the data? If not, then I'd think good and hard about whether you want to spend £500 on backups. (And rethink that exercise at £100, £50 and £1,000 too)

One other think to consider is fan noise. A NAS is usually on 24/7. If you don't have somewhere to stick it where the fan won't bother you then you'll want to look at fanless models and that will restrict your choice a lot.
 
JonathanRyan said:
It probably does though it's quite an old one (TS-109 - not the "pro") - one day when I'm not busy I'll spend some time and figure it out. The QNAP is basically an iTunes library and it seems noticeably quicker accessing via the Sonos system than from iTunes on the Mac.


snip....
One way to look at it is.......if your drive failed tomorrow, would you pay me £500 if I could guarantee to recover the data? If not, then I'd think good and hard about whether you want to spend £500 on backups. (And rethink that exercise at £100, £50 and £1,000 too)
Click to expand...

If it was my only copy, without hesitation!

JonathanRyan said:
One other think to consider is fan noise. A NAS is usually on 24/7. If you don't have somewhere to stick it where the fan won't bother you then you'll want to look at fanless models and that will restrict your choice a lot.
Click to expand...

I am going to be putting the drives in a spare room, so noise shouldn't be an issue.

Re time spent backing up, hopefully when it is set up i hope it will be an almost invisible process, or am i being naive?
 
neil_g said:
QNAP is a pretty well respected brand, id expect it to do AFP as standard.

might be worth checking your ethernet adapter on the mac has detected the speed of the link correctly.
Click to expand...

Possibly worth mentioning that OS X 10.9 Mavericks now supports SMB2; indeed SMB2 is default network protocol for file sharing in Mavericks. It ought to provide better throughput than AFP.

AFP, SMB (v1) and NFS are still supported in 10.9.

http://arstechnica.com/apple/2013/10/os-x-10-9/22/#sharing

Also, even if you're keeping your RAW files on the NAS, keep your catalog file(s) on a local drive. Lightroom won't permit you to keep them elsewhere.
 
Musicman said:
Possibly worth mentioning that OS X 10.9 Mavericks now supports SMB2; indeed SMB2 is default network protocol for file sharing in Mavericks. It ought to provide better throughput than AFP.

AFP, SMB (v1) and NFS are still supported in 10.9.

http://arstechnica.com/apple/2013/10/os-x-10-9/22/#sharing

Also, even if you're keeping your RAW files on the NAS, keep your catalog file(s) on a local drive. Lightroom won't permit you to keep them elsewhere.
Click to expand...

yup, always keep the Cat local. on SSD if you can.

lets hope OSX over SMB2 isnt a POS like SMB.
 
Musicman said:
Possibly worth mentioning that OS X 10.9 Mavericks now supports SMB2; indeed SMB2 is default network protocol for file sharing in Mavericks. It ought to provide better throughput than AFP. AFP, SMB (v1) and NFS are still supported in 10.9. http://arstechnica.com/apple/2013/10/os-x-10-9/22/#sharing Also, even if you're keeping your RAW files on the NAS, keep your catalog file(s) on a local drive. Lightroom won't permit you to keep them elsewhere.
Click to expand...

Indeed and I can confirm that the MacBook Air (2012) with OS X Mavericks (10.9) seems to work flawlessly over SMB 2 with a QNAP TS-421 that I am setting up for a client.

If you are using a QNAP, consider accessing it over iSCSI. This is a highly efficient protocol, which is used for smaller SAN-like setups. To give you an idea, a client of mine a few years back required a temporary and cost-effective storage solution for a 200 device Mac and PC network. I installed two QNAP 6-bay devices, with trunked network ports (bonding the two network ports to give 2 Gbps). The performance was outstanding for the price.
 
Musicman said:
Possibly worth mentioning that OS X 10.9 Mavericks now supports SMB2; indeed SMB2 is default network protocol for file sharing in Mavericks. It ought to provide better throughput than AFP.

AFP, SMB (v1) and NFS are still supported in 10.9.

http://arstechnica.com/apple/2013/10/os-x-10-9/22/#sharing

Also, even if you're keeping your RAW files on the NAS, keep your catalog file(s) on a local drive. Lightroom won't permit you to keep them elsewhere.
Click to expand...

Yep, catalog and previews will all be on local drive.
 
neil_g said:
if you want link aggregation remember youll need a compatible switch.
Click to expand...
Too true, but not a great expense for a simple managed switch.. In fact, probably a worthwhile investment, as most of the NAS setups I have seen running over broadband routers do not run to their best.

Also, if you are storing large photos, audio or video, you could benefit from enabling jumbo frames. An 8-port Web Managed HP Procurve can be had for not a lot of money these days.
 
Last edited:
Slightly off topic....but is there a good resource where I can learn how to troubleshoot my network?

I don't think it's running fast enough but I don't know why. I also don't have any hard data other than a feeling that it should be faster.
 
JonathanRyan said:
Slightly off topic....but is there a good resource where I can learn how to troubleshoot my network? I don't think it's running fast enough but I don't know why. I also don't have any hard data other than a feeling that it should be faster.
Click to expand...
Give us a rundown Jonathan of your setup (router/switch, computers etc)?
 
MarcM said:
Give us a rundown Jonathan of your setup (router/switch, computers etc)?
Click to expand...

2012 iMac running ML

Zyxel 8 port gigabit switch (http://www.dabs.com/products/zyxel-8-port-desktop-gigabit-ethernet-switch-8J1M.html) - everything goes through that

Hung off the switch is a BT Home Hub feeding fibre and wifi

There's also a spur that runs to a cupboard where there's a Netgear 8 port Gigabit switch. Plugged into that are an Apple Time Capsule and a QNAP TS-109 which is basically an iTunes library (NOT server).

Off the Zyxel is another spur (about 30 metres long and buried underground) which connects to a 4 port gigabit switch (can't remember the brand and it's dark out there) which runs Apple TV in the gallery and a ReadyNAS NV+ 4 bay RAID backup.

Any wifi devices (laptop, tablets, phone) use the BT wifi. Anything else (PC, TV, DVD player, Sonos system) run via the Zyxel.

Sonos has its own mesh wifi too.

Pretty basic ;)
 
neil_g said:
if you want link aggregation remember youll need a compatible switch.
Click to expand...
And you'll need to check the protocol. You will find that 2 x network ports from 1 computer to another will only run at 1Gbps - the selection of which port it goes out is normally done by the IP address of the destination, meaning you'll always be using one port between two fixed points.
 
OK, the first thing you should do is get some sort of benchmark. Have a look at this page http://www.smallnetbuilder.com/lanw...ur-network-five-ways-to-measure-network-speed and use one of the tools to test the speed between two devices connected directly to the Zyxel

Do the same on the other switches to see what speeds you are getting and wether any are under-performing. You can then run end-to-end tests, e.g from a device connected to the Zyxel to a device connected to the the Netgear (use the same two devices, such as two laptops if you can). These tests may throw up any obvious issues such as a faulty LAN cable, device or LAN port.

The steps above are only a starting point, as their are too many variables which could be causing issues, but it should point you in the right direction if there are problems.

Something I would say is that the layout (or topology) of your network is not ideal, especially with the switches etc you are using. If at all possible, I would look at optimising the basic infrastructure, as until you do this, you're always likely to get less than satisfactory performance.

1. A better setup if you can would be to replace the Zyzel at the heart of your network with a HP Procurve (the best switches at anywhere near their price point IMHO). Take your pick out of the 1810G range for a decently priced managed gigabit model.

2. Try to avoid running switches off of the main one. Whilst the HP will handle the traffic better, it is best to avoid this with lower grade switches (I include the Netgear in this), as performance suffers.

3. If possible, run all of your Ethernet cables back to the main switch. If you must run an auxiliary switch, get another HP (a cheaper unmanaged one if necessary).

4. Get a proper dedicated wireless access point, or at least get a decent router with built in wireless to replace the Home Hub.

Hope this helps.
 
MarcM said:
Hope this helps.
Click to expand...

Helps a lot - thank you.

It's going to be a while before I can try all that but I'll be back ;)

Running extra cables is unfortunately not an option. We ran Cat5e under the floors (and garden) during a rewire and now the floors are down to stay ;)

I have to admit, I never expected to see the day when I had to use a 4 port switch to run my TV.......
 
FYI: I use TP Link managed switches rather than the HP ones. The HP ones look like they have fans in which is a big no-no for me. Plus they are more expensive... I use a TL-SG2216 and TL-SG3216. The 2216 will be fine for what you want and can be had for less than £90 delivered. If you want smaller, they do a 3210 8 port managed switch, but it is almost the same price as the 16 port switch.

The difference between the two is a few less (corporate level) menu points in the interface.

I'd also replace the HH with an Asus RT-N66U (or AC66U if you want access to AC wireless).
 
PS: iperf is the simplest tool to troubleshoot networking. You use it as a client/server and can alter both buffer sizes used for network transmission (not the packet sizes used on the network) and number of threads used. It can be quite an eye opener!
 
Each to their own, but I wouldn't go to trouble of replacing what you have already with TP-link gear.

The whole HP Procurve range come with a lifetime warranty (which they honour) and there is good reason why they offer this - their kit is rock solid.

The 8 port 1810G can be had for £70, so will not break the bank for someone relying on their network professionally. The 24-port can be had for around £140.

I've just installed five of the 48-port models, along with some relatively expensive 10 Gig 2910's. They have (as with all HP Procurve kit) been rock solid.

The networks I used to build usually consisted of about £1/4 million worth of Cisco kit. If budgets were low, but a high port count was still needed, it would be HP all the way. Pretty much an industry standard approach, because it could be relied upon. Brands like Netgear, D-link and the like wouldn't get a sniff!

No fans in the 8-port IIRC. The five 48-ports I installed are very quiet, so fan noise is not an issue even if there is one.

I'm not saying that cheaper stuff won't work, it's just that I've seen to much to know that saving a few quid here and there just isn't worth it.
 
Last edited:
define professionally though. 1/2 machines connecting to a NAS a corporate grade switch isnt really necessary, hell i managed fine on a GS108. if youre connecting 24-48 gig devices then maybe spend more cash.
 
MarcM said:
The whole HP Procurve range come with a lifetime warranty (which they honour) and there is good reason why they offer this - their kit is rock solid.
Click to expand...
As does the TP-link warranty on the SMB stuff (which all the managed switches are).

I'd be happier with your dissing of TP-link if you had issues with them or that they are proven more unreliable/slower/buggy - they are generally regarded as a good, reliable budget brand. Saying HP are better with no reason why other than hearsay is like saying WD are better drive manufacturers than Seagate....
 
arad85 said:
PS: iperf is the simplest tool to troubleshoot networking. You use it as a client/server and can alter both buffer sizes used for network transmission (not the packet sizes used on the network) and number of threads used. It can be quite an eye opener!
Click to expand...

Downloaded it, run it. I have literally no idea what I'm looking at.

One day I'll pay somebody to sort all of this out. But in the meantime, my AppleTV has stopped working and that means getting a ladder out of the shed.
 
Calm down chaps, no one is dissing TP-link Kit, I have used one of their routers in the past.

I'm just offering a user my own perspective from years of building networks for companies large and small.

Cheap-brand small switches have caused many issues in networks I have seen. They are often fine when they work, but when they fail, they can cause havoc (random packets, ARPing, high packet loss etc). All tricky for the novice to diagnose.

This is my experience which may differ to others. Having managed IT in banking, aviation, education and many other markets, I've yet to see a Netgear or TP switch!
 
Last edited:
arad85 said:
As does the TP-link warranty on the SMB stuff
Click to expand...

Five years after product being discontinued. May as well be lifetime I know, but worth pointing out.

I'd be happier with your dissing of TP-link if you had issues with them or that they are proven more unreliable/slower/buggy - they are generally regarded as a good, reliable budget brand. Saying HP are better with no reason why other than hearsay is like saying WD are better drive manufacturers than Seagate....
Click to expand...

See my post above. This isn't assumption, 24 years IBM, AT&T, NCR, Computercenter, Sun etc etc... cheap small switches cause all kinds if issues. HP used throughout all industries because they perform well against other high end equipment. Seagate and WD are comparable at all market levels. Comparing HP with TP is like comparing Audi with Vauxhall.
 
You must log in or register to reply here.
Back
Top