Solved - copying from memory cards slows down drastically

Messages
996
Name
John Stewart
Edit My Images
Yes
TLDR; not all fast SSD drives are equal.

An issue I’ve had on various PCs and laptops (all windows based) for several years has been when copying large amounts of data from memory cards (mostly raw files) is that the copy speed initially starts high, but then after a minute or two it slows down and at times almost stops or pauses, before speeding up briefly then slowing down and eventually settling down to a copy speed much slower than the card/USB interface is capable of.

The problem got worse when using faster memory cards such as CFast and CFExpress. I thought it was perhaps thermal throttling of the memory cards as the Cfast ones get quite toasty and tried various things like updating USB drivers, disabling windows indexing on the target drive, switching off the virus checker during copying etc. Though these all helped in a small way by increasing the maximum transfer rate, the copy speed would still drop after a couple of minutes.

Thinking about the issue last week after a nightmare trying to transfer photos and videos between BTCC races, it occurred to me that I never had the issue when using slower CF cards, as the write speed to disk was higher than the read speed from the cards. But surely the write speeds of NVME SSDs are much higher? Running tests on them confirmed that the internal SSD drive on my laptop had write speeds of over 3000 Mb/s, and when copying from a Cfast card the copy speed was about 1/10th of this. So where was the issue?

Something I wasn’t aware of is that most modern SSDs have a finite amount of cache on board, sometimes this is DDR4 and sometimes slower memory. When writing large amounts of data over a period of minutes, this cache eventually fills up. At this point the on board controller either stops accepting new data (for up to 30-60 seconds), or slows down dramatically, matching the actual write speed to the drive storage. Many reviews of SSDs don’t detail the amount of cache on board, and what happens when this fills up during sustained writes as they are more focussed on the headline read/write speeds. I did find one site which routinely performs these tests, and the difference between drives with similar maximum read/write speeds under sustained writes can be staggering. Some drop to a fraction of their maximum speeds, whilst others have a much higher sustained write speed. The intel 670p SSD that my laptop came with falls into the former category. It’s very quick when handling smaller reads/writes, but when writing 200Gb of data to the drive, it slows to a crawl after a minute of so.

Even though my laptop uses the older pcie3 interface, I thought that some of the better pcie4 drives might offer better sustained write speeds even running at half speed, and after reading some reviews I found a few drives which promised much better sustained write performance. £62 from Amazon got me a 1TB Seagate Firecuda 530 drive. I’m aware you can get similar capacity drives for up to £20-£30 less, and even lesser known 2TB drives for the same money. However, they didn’t have the sustained write speed I was being frustrated with.

I cloned the existing drive and installed it in my laptop a short while ago. I still had the files on the CFExpress card from last weekend, and repeated the same copy. Wow, what a difference! What took a frustrating 20+ minutes or so last weekend completed in less than 90 seconds, maintaining a steady 745Mb/s all the way through. This is likely the real world limit of the USB port or memory card. Needless to say I’m over the moon, and I’m now thinking about swapping out the SSD in my desktop PC as well, which is also an older pcie3 interface
 
Well done for figuring out the problem. It sounds like it will be transformative

What is the site that had the tests? It would be interesting to see how the NVME drives in my PC measure up.
Servethehome

generally the sustained write tests are on page 3. There are other drives which offer similar or better performance, but for laptop use I also considered temperature as some can run very hot, consume more energy or were more expensive for the same size of drive.
 
Some of the more technical tech sites, such as Tom's Harware, will include sustained read and write tests in their storage reviews to cover exactly this kind of issue

e.g.


Scroll to 'PC Sustained Write Performance and Cache Recovery' for the graphs
 
Some of the more technical tech sites, such as Tom's Harware, will include sustained read and write tests in their storage reviews to cover exactly this kind of issue

e.g.


Scroll to 'PC Sustained Write Performance and Cache Recovery' for the graphs
Yes, Toms Hardware does decent in depth reviews and was one of the sites I used. Many don’t though, including this review of my 670p which doesn’t even hint of the issue I encountered: https://uk.pcmag.com/ssds/132103/intel-ssd-670p

I suppose copying large amounts of files is a niche use case though compared to most users who are more concerned with load times for games etc.
 
I suppose copying large amounts of files is a niche use case though compared to most users who are more concerned with load times for games etc.
I think when NVME's were used mainly as boot disks, sustained write speed was not that important, but now they are replacing HDD's and SATA SSD's in some PC builds, the sustained write speed starts to become important as you have found.

In my new PC I moved my images from a internal 2TB HDD to a 4TB NVME. The copying of all the files took quite awhile, as though the NVME drive is very fast, the speed was limited as to the read speed of the HDD. :rolleyes: :LOL: Speed is always the slowest link in the chain. ;)

Thanks for posting the link btw, it's an interesting site. :)
 
i think the QLC drives are the worst for this, they ezhaust the cache and crawl
 
I have heard that there are some configurations of new Macs that come with slower memory, so it is definitely something to research thoroughly before buying a new computer.
 
Back
Top