The future of storage -- what options exist when it comes to SSD arrays?

[drumstyx]

I'm constantly floored by the rapidly dropping prices of SSDs, to the point where I've flippantly bought SSDs for various machines I've got laying around. $35 CAD for a branded 250gb NVMe? Good lord it's cheap now.

 

That got me thinking about the future of storage -- SATA is no longer being developed any faster, which means 2.5" SSDs will go by the wayside in favour of M.2, but that's a whole other issue of bus limitations, and I'm here to talk about storage. Hard drives don't seem to be getting much cheaper. Of course, it's happening slowly, but here in Canada, a GREAT deal on an 8TB drive (the cheapest per TB right now here, as opposed to the 12TB deals y'all are getting in the USA these days) is $180 for a WD drive that still needs to be shucked. That's $22.50/TB. An extremely cursory search shows a Silicon Power 1TB SATAIII SSD at $115 on Amazon. A cursory Black Friday deal search comes up with a Team Group 1TB SATAIII SSD at a shocking $90 from Canada Computers! So we're at a factor of 4-5x difference, and just a year ago the factor was more like 8-9x. Point is, even assuming a non-linear change year over year, we can probably expect a crossover some time in 2021-2023 -- which is the timeframe we all should be planning replacements/growth for anyway.

 

So now the tech issues: SSDs need to be TRIMmed periodically, which as I understand it, deals with some of the drawbacks associated with wear-leveling. This means parity calculation as we know it is fundamentally broken for SSDs as long as they require TRIMming.

 

My main question is: Can this be rectified, or does the very concept of parity need to be revised? If so, what options exist for this right now? What options WILL exist? Will a mix-n-match system like Unraid even be possible? What's on the roadmap here?

[testdasi]

I think you have a misunderstanding on the role of TRIM.

• TRIM allows the OS to tell the SSD which blocks are effectively empty i.e. data on them can be ignored.
  • This allows the SSD, when writing to such blocks, to write a new block instead of needing to read what's on it first before writing. This is how TRIM helps maintaining the SSD performance over time.
  • On its own, the TRIM command does nothing to affect wear leveling or endurance or parity calculation.
• What affects parity calculation is garbage collection, which is essentially SSD defrag.
  • Theoretically, the OS isn't aware of how data is moved during garbage collection, so any data that is moved would render the parity invalid (for those blocks).
  • As per what johnnie observed, it strangely only seems to affect a small number of SSD's in practice (while theoretically, it should affect all SSD's). That can be explained away if physical and logical block addresses are maintained independently.
    • Simplistic example: John, Jack and Jim live in house number 1, 2, 3 respectively. They are forced to change house (change of physical address) so now live in house number 2, 3, 1 respectively. However, if you don't look for house numbers but rather the signs in front of the house to look for John's house (same logical address) then it doesn't matter.
  • It could also be that the SSD actually tells the OS what data has been moved where and thus parity was calculated in the background and few SSD's actually do this silently.
    • Or it could be that GC just don't run in the majority of johnnie's SSD's - kinda unlikely I would assume
• With TRIM, the SSD knows which blocks are empty and thus don't do garbage collection on those blocks. That helps with wear level (and thus improves endurance).
  • Using the simplistic example above, it is like you are told John's house is empty (or house number 1 is empty) so you don't have to go look for John to move house.
• So no, SSD's do NOT need to be TRIM'ed periodically and TRIM, on its own, does not invalidate parity.
  • Depending on the exact code, it could be that TRIM is used to trigger GC (kinda unlikely scenario in my mind but possible), in which case, under the silent GC case, TRIM does invalidate parity but it's a very specific case, not a generalized statement.

I believe the reason why Unraid has never officially supported SSD aray is because LT has not done any major testing to assess the scale of any (potential) issue. It would be highly irresponsible for LT to rely only on a single user's anecdotal story. "Trust, but verify" so to speak.

 

However, because of the above points, I would expect SSD array to always come with a caveat that some SSD's can cause parity error.