Data strategy for video work?

Messages
3,034
Name
Tim
Edit My Images
Yes
Expanding from photo to video over the past couple of years has resulted in a mushrooming of data. My question is what's the best strategy to manage the data?

For photos, my approach is to cull from the cards on ingest to a working drive and make backup copies. With decent working drives and 8TB of NAS for backup copies no real problem.

For video, I'm adapting the process as follows:

1. Ingest
Similar cull on ingest, although not as straightforward to make selections. Working drive needs to be SSD for editing and rendering speed and with the large file sizes the SSD fills up fast. So it means I copy to SSD working drive and then to two backup drives.

2. Create proxy files
To improve editing speed I create proxy files. I'm treating these as temporary so make only 1 backup copy while working on the edit.

3. Edit
Creates new editor files which are small xml files.

4. Render product
Rendering creates more big files, full 4k, 1080p, compressed versions for different platforms etc. Similar to proxy files I treat most as temporary as can be recreated so 1 backup copy.

5. Delete proxies and compressed output
Once the project is complete, delete the proxies, compressed rendered files and everything off the SSD. This leaves 2 backup copies of the original ingested files, editor project files, and uncompressed rendered output.

Does this make sense, any ways to improve or other ideas? The backup copies are in different places, I may move one online, any suggestions for that? I'm up to over 50TB and it's growing all the time.
 
Guess, as with photography, it depends on what type of video work you are doing.. and what purpose.

For me - it is sports, that's player interviews, features, game highlights, etc.
Some projects are for clients who just need the end product, no need to keep all the B-roll and unused clips.
So after delivery - I just keep the rendered version in archives and delete the rest.

For games/training footage.

Edit the project to client - deliver it.
Then with the clips that are usable - I rename them to show which players are on the clip and what type of action is happening
For example (as I mainly do basketball) : LeBron James - 2p driving dunk.mp4
I do no edits/adjustments to these, as they are the original files and this way I can easily find them if/when needed to do something with them in the future.

After going through all the files - the ones that didn't get named.. had nothing useful on them or were OOF. So they get deleted...

Two back-ups on separate external storage ...
 
Guess, as with photography, it depends on what type of video work you are doing.. and what purpose.

For me - it is sports, that's player interviews, features, game highlights, etc.
Some projects are for clients who just need the end product, no need to keep all the B-roll and unused clips.
So after delivery - I just keep the rendered version in archives and delete the rest.

For games/training footage.

Edit the project to client - deliver it.
Then with the clips that are usable - I rename them to show which players are on the clip and what type of action is happening
For example (as I mainly do basketball) : LeBron James - 2p driving dunk.mp4
I do no edits/adjustments to these, as they are the original files and this way I can easily find them if/when needed to do something with them in the future.

After going through all the files - the ones that didn't get named.. had nothing useful on them or were OOF. So they get deleted...

Two back-ups on separate external storage ...

Thank you this is helpful. I'm going through the archive again to see what can be deleted. Next step is to replace the box of disc drives with a RAID.
 
Making progress with the next step, the new unraid server is configured and ready to go and I've started the process of migrating all data from the box of external drives. It's going to take quite a while to get organized, deleting what I don't need anymore then updating the backup too. Any have ideas on tips on the best way to approach cleaning up the old data?
 
That looks like a decent strategy but it might be worth considering,

If your video work is for commercial clients then I think it's important you agree how the data is managed. If your client is expecting you to store huge amounts of data indefinitely then I think they should contribute to the cost of of the storage. Lots of the jobs I shoot have no value to me outside of the job and the content is likely to be relatively short lived. If the client doesn't want me to keep the data I will keep a good copy of the finished render and delete everything else.

Also it's very tempting to oversupply the quality of the original footage. In the past I have spent days shooting hundreds of gig of 4k 50fps BRAW only to see it squashed to a 1 min video for use on facebook and 1080p would have been more than adequate. If your client wants 4k rather than 1080p then that's no problem but in most cases good 1080p is more than enough. I now agree what footage they want and if they want full 4k workflow from capture to delivery I will quote more due to all the extra storage and file handling costs.

On a side note I use Premier pro and I haven't found any appreciable difference in editing and rendering performance between work hosted on ssd and regular fast hd's.
 
Back
Top