NVMe cache went bad in SMART test but I cannot see why?

[testdasi]

12 minutes ago, Diggewuff said:

Thanks for your detailed Explanations. Downloads and Plex Transcods cannot be the main reason for the nearly 400 TBW.

Is it possible to over provision any SSD Myself?

I don't have any VM vdisks and my Docker Image is about 100gig so 500gig of usable space is plenty enough for my use. Switching to 1 tb or let's say overprovisioned 840gigs would already be a huge upgrade for me but I don't want to lose the speed of NVMe.

I cannot find any fast NVMe drives that are rated for significantly higher TBW.

Write amplification can theoretically explain the 400 TBW (or at least a major part of it).

• Plex transcode and downloads writes are mostly random, which increases write amplification (all other things being equal). Essentially when a random write is required, the controller must pick a block to write within a reasonable amount of time (remember SSD is all about response time, even more so with NVMe). If it can't figure out a good block to pick, it has to erase some blocks, move data around, rewrite data that has already been written etc..
  • So to reduce WA, you need another factor (i.e. no longer "all other things being equal") to come into play. The 2 most common factors are over-provisioning and trim. The former relies on the controller to be aware of the over-provisioned space to write to and the latter makes the OS aware of "good" area to write. Both has the same commonality, which is available free space. Over-provisioning forces an area on the chip to always be free and trim basically marks free area as "good".
• Wear leveling also increases WA. WL essentially moves data out of a less-used area into a more-frequently-used area to free up the less-use areas for write i.e. so blocks are more evenly worn out.
  • That's why static data on the SSD inadvertently increases WA but the degree of which depends very much on the WL algorithm.

So when you put 2+2, it's obvious that the more free space and the less static data, the less WA there will be.

 

As I mentioned 512GB SSD is not at all that large. If you use it for temp data, it will eventually run out of idea as to what blocks are "good" to write to, which requires trim to reclaim. People typically run trim weekly, which isn't often enough if, for example, one writes 1TB of data daily. You can also over-provision the SSD yourself (which I call soft over-provision) by simply leaving as much free space available as possible. Some controllers are smart enough to detect such free space and use it automatically as over-provisioned space. Even assuming a dump controller, probability alone can make the difference.

 

Your last sentence is the reason why I said I think you approached the issue from the wrong angle. Without paying an arm and a leg for enterprise-level solutions (e.g. SLC SSD - do they even sell these any more?), you really can't do better than Samsung SSDs in terms of endurance rating.

 

By the way, it sounds alarming but certainly don't be alarmed. Your 960 is more likely than not to still last a while. I have tried to purposely run an SSD to the ground and it is way harder than people think, as long as it's not an Intel SSD.

[Diggewuff]

• Trim was always running daily on my SSD.
• Can I improve anything about WL?
• Would you rather suggest buying a bigger consumer SSD than a better rated smaller enterprise SSD to just leave space free and to hope for the controller to make good use of it and thereby reducing the WA?
• About 180-200€ for a Samsung PM983 960GB M.2 seams reasonable for me. Would you consider that as enterprise grade or would rather buy a Samsung 970 Evo M.2 1TB for about the same price?

Thank you very much for sharing your expertise on that.

[testdasi]

1 hour ago, Diggewuff said:

• Trim was always running daily on my SSD.
• Can I improve anything about WL?
• Would you rather suggest buying a bigger consumer SSD than a better rated smaller enterprise SSD to just leave space free and to hope for the controller to make good use of it and thereby reducing the WA?
• About 180-200€ for a Samsung PM983 960GB M.2 seams reasonable for me. Would you consider that as enterprise grade or would rather buy a Samsung 970 Evo M.2 1TB for about the same price?

Thank you very much for sharing your expertise on that.

Assuming you have a realistic expectation to claim warranty with Samsung for enterprise SSD (that assumption cannot be assumed!), it comes down to how much write you expect to do.

• The PM983 is rated for 1.3 DWPD for only 3 years = 0.96TB * 365 * 3 = 1051TB TBW.
• The 970 Evo has a 5-year warranty but only for 600TB TBW (= 600 / 5 / 365 = 0.329 DWPD).

In other words, do you expect to write 600TB over 5 years or 1000TB over 3 years?

 

Also Note that the PM983 is 22110 and the 970 Evo is 2280. Not all motherboards support 22110.

 

I have both the PM983 and 970 Evo. They have the exact same ID under the devices tab suggesting they have the exact same controller. I can't say for sure that they are fundamentally the same; however, given the same controller and similar price point, my hunch is the PM983 is simply the 970 Evo with a hard 4% over-provisioning (which is why 960GB vs 1TB) and a redesigned form-factor.

 

With regards to WL, you have no control over that. It's the manufacturer algorithm. The best you can do is, where possible, split write-intensive and read-intensive into different drives.

[Diggewuff]

5 hours ago, testdasi said:

Assuming you have a realistic expectation to claim warranty with Samsung for enterprise SSD (that assumption cannot be assumed!), it comes down to how much write you expect to do.

• The PM983 is rated for 1.3 DWPD for only 3 years = 0.96TB * 365 * 3 = 1051TB TBW.
• The 970 Evo has a 5-year warranty but only for 600TB TBW (= 600 / 5 / 365 = 0.329 DWPD).

In other words, do you expect to write 600TB over 5 years or 1000TB over 3 years?

Let’s take 5 years as an example. From my calculation TBW will be at about 1000 maybe a bit less because of WA and the larger drive size. 

970 will be way beyond the specified TBW. But still in warranty. SMART Test will again be failing because of percentage used over 100%. I‘ll again have no idea if the drive fails or not even if it still works. 

PM983 will be still in spec for TBW but out of warranty because of the time. Percentage used and SMART Test will be ok but no waranty. 

 

Usually I claim warranty when there is any but  if the claim process is costly and the prospects of success are low. I just leave it. Wich is my experience for Samsung consumer drives. No idea if this will be different to Samsung Enterprice drives. Advance RMA maybe? WD offers that. 

 

What Formfactor goes: My MB accepts 22110. 

 

As you have both drives installed: 

Do you encountered any differences in read or write speads?

How is each percentage used value scaling to your TBW values? Are they indead scaling different as I assumed above?

Could you maybe post those values for each drive as a reference?

 

[testdasi]

On 7/30/2019 at 9:53 PM, Diggewuff said:

As you have both drives installed: 

Do you encountered any differences in read or write speads?

How is each percentage used value scaling to your TBW values? Are they indead scaling different as I assumed above?

Could you maybe post those values for each drive as a reference?

My PM983 is slower in CDM. However, it's a 3.84TB model so it might very well be due to the larger capacity (same controller + larger capacity = generally slower). In real life though, I have not noticed any diff.

In terms of percentage used, both of mine are still 0% so I have no basis to assess these 2. For the Intel 750 I have though:

• Intel 750 1.2TB A
  • actual 104 TBW / rated 127 TBW (yes, 127!) = 82%
    • The 750 has the same TBW rated for 400GB, 800GB and 1.2TB models, which doesn't make sense, so assuming the rating was conservatively done based on the 400GB then: actual 104 TBW / rated 381 TBW = 27%
  • percentage used = 3%
• Intel 750 1.2TB B
  • actual 47 TBW / rated 381 TBW = 12%
  • percentage used = 1%
• Considering 104 = 2.2 x 47, it kinda matches 3% vs 1% (taking into accounts rounding diff).
  • Based on that, the more realistic TBW rating for the 750 should be about 4000TB (or about 1.8 DWPD for 5 years)

So I guess the conclusion is actual percentage used and actual TBW scales rather well, at least for the same model.

However, you can't use the manufacturer rated TBW to estimate your expected percentage used. Manufacturers just don't necessarily rate their drives realistically.

 

I notice that none of my SSD appears to exhibit your rather extreme level of write amplification.

The read:write ratios are:

1. Intel 750 A 1.02
2. Intel 750 B 1.04
3. Samsung 970 EVO 2.04
4. Samsung PM983 3.81

(1) and (2) spent a significant portion of their life as my cache drives; however, I am very particular about separating write-intensive and read-intensive data (e.g. the PM983 is almost exclusively write-once-read-many while the cache is most of the time write-once-read-once), as well as following good practices e.g. regular trimming, soft over-provisioning etc.

So at least anecdotally, I very much trust my methods.

 

 

 

CDM bench:

PM983 3.84TB

   Sequential Read (Q= 32,T= 1) :  3043.498 MB/s
  Sequential Write (Q= 32,T= 1) :  1433.328 MB/s
  Random Read 4KiB (Q= 32,T= 1) :   291.462 MB/s [ 71157.7 IOPS]
 Random Write 4KiB (Q= 32,T= 1) :   264.787 MB/s [ 64645.3 IOPS]
         Sequential Read (T= 1) :  1566.127 MB/s
        Sequential Write (T= 1) :  1392.059 MB/s
   Random Read 4KiB (Q= 1,T= 1) :    24.914 MB/s [  6082.5 IOPS]
  Random Write 4KiB (Q= 1,T= 1) :    64.947 MB/s [ 15856.2 IOPS]

970 EVO 2TB

   Sequential Read (Q= 32,T= 1) :  3543.933 MB/s
  Sequential Write (Q= 32,T= 1) :  2499.500 MB/s
  Random Read 4KiB (Q= 32,T= 1) :   282.467 MB/s [ 68961.7 IOPS]
 Random Write 4KiB (Q= 32,T= 1) :   239.125 MB/s [ 58380.1 IOPS]
         Sequential Read (T= 1) :  1893.878 MB/s
        Sequential Write (T= 1) :  2411.367 MB/s
   Random Read 4KiB (Q= 1,T= 1) :    37.470 MB/s [  9147.9 IOPS]
  Random Write 4KiB (Q= 1,T= 1) :    79.055 MB/s [ 19300.5 IOPS]