Inconsistent SSD Speed Test

[TODDLT]

I have a 500 GB Samsung 850 EVO SSD in my array.  The drive was purchased in August of 2015.  During recent parity checks I did catch a few very sharp drop off and recovery periods in the early stages of the parity / rebuild operations.  I'm talking about speeds dropping from 190 MB/Sec down to 60-70 MB/sec.  These drops lasted for a 2-3 minutes and then recovered.

 

I ran a speed test today and please see the attached graph for this SSD.  Very slow overall and wavering.  I also included a Smart Test report ran today.

 

I have a 2nd drive, same model purchased in November of 2017, and it shows much faster and consistent speed results.

 

Any ideas what is going on?  Is this drive failing? 

I did turn off all my dockers (Plex related) except the speed test.  Nothing other than core unraid functions should be accessing the drive.

 

Samsung_SSD_850_EVO_500GB_S21HNXAG454345K-20181004-1913.txt

 

 

Edited October 4, 2018 by TODDLT

[JorgeB]

I remember the 840 EVO suffered from an issue where it would get slower with old data, firmware update "fixed" most of the problem by continually re-writing data when it get to a certain age, maybe the 850 suffers from a similar issue, you could try manually re-writing the data there to see if it makes any difference.

[TODDLT]

20 minutes ago, johnnie.black said:

I remember the 840 EVO suffered from an issue where it would get slower with old data, firmware update "fixed" most of the problem by continually re-writing data when it get to a certain age, maybe the 850 suffers from a similar issue, you could try manually re-writing the data there to see if it makes any difference.

How do you manually re-write data, or know what to re-write.  There are 52K files on this drive with 120 GB of data.  Do I have to copy it off and back on?

 

EDIT, I guess the 2nd question is do you think there is any way this is a sign of a failing drive?

Edited October 5, 2018 by TODDLT

[TODDLT]

1 hour ago, johnnie.black said:

I remember the 840 EVO suffered from an issue where it would get slower with old data, firmware update "fixed" most of the problem by continually re-writing data when it get to a certain age, maybe the 850 suffers from a similar issue, you could try manually re-writing the data there to see if it makes any difference.

Continuing to think about this issue....

 

If the issue were the age of the data on the drives, why would this show up on a speed test.  I'm assuming the Speed Test (Disk Speed Docker), would be reading / writing it's own data, not just reading what is already on the drive.   Is this true?

 

Attached is now the comparison between the new and old drives.  Again, my biggest concern is if this is a pre-fail indicator.   What else can I look for on an SSD?

 

[TODDLT]

I ran a speed check every 2%  the previous runs were every 10%.

 

I think you can actually see where the tips drop below 180 ish and those are probably the places it was dragging the parity check down.

 

Edited October 5, 2018 by TODDLT

[JorgeB]

7 hours ago, TODDLT said:

How do you manually re-write data, or know what to re-write.

Move some data out, then move it back in, or write a large file to it and redo the test.

 

6 hours ago, TODDLT said:

I'm assuming the Speed Test (Disk Speed Docker), would be reading / writing it's own data, not just reading what is already on the drive.   Is this true?

No, it just reads the existing data.

[TODDLT]

8 hours ago, johnnie.black said:

Move some data out, then move it back in, or write a large file to it and redo the test.

 

its a lot of small files but I'll try to take a full directory out and back, see how that works out.

 

Do you know if that issue was fixed in the 860 series drive?  You said there was a firmware update that fixed it in the earlier versions?

 

Thanks for the help as always!

[TODDLT]

8 hours ago, johnnie.black said:

Move some data out, then move it back in, or write a large file to it and redo the test.

 

What if I took the drive out of the array., formatted it with unassigned devices, and then let the full set of data re-build?

 

[JorgeB]

20 minutes ago, TODDLT said:

Do you know if that issue was fixed in the 860 series drive?

It was supposed to be fixed on the 850, but seems very similar to the 840 problem, hence my suspicions.

 

15 minutes ago, TODDLT said:

What if I took the drive out of the array., formatted it with unassigned devices, and then let the full set of data re-build?

Rebuild will also work as a test, and no need to remove the SSD from the array, just unassign it, start the array, re-assign it and start once more to rebuild.

[TODDLT]

1 minute ago, johnnie.black said:

It was supposed to be fixed on the 850, but seems very similar to the 840 problem, hence my suspicions.

 

Rebuild will also work as a test, and no need to remove the SSD from the array, just unassign it, start the array, re-assign it and start once more to rebuild.

OK will try one or the other tonight and let you know.

 

Thanks for the time.

[TODDLT]

8 hours ago, johnnie.black said:

It was supposed to be fixed on the 850, but seems very similar to the 840 problem, hence my suspicions.

 

Rebuild will also work as a test, and no need to remove the SSD from the array, just unassign it, start the array, re-assign it and start once more to rebuild.

OK I did the complete drive rebuild when I got home, then speed test.

1st. it took 1 hr 50 min to do a 500 GB drive rebuild.  It started out saying it would take about 50 minutes and then increased obviously as the drive speed dropped off.

The speeds started up around 170 MB / sec but then dropped to as slow as 30 mb / sec and stayed down low for a good part of the re-build.  I also noticed the older drive spiked up to 44 degrees much faster than the newer drive.

I ran the disk speed test on both 850 EVO drives and here are the results.  

 

Below is drive 1 (older).  The wavy lines are all the test results from yesterday.  The solid line at about 225 MB/Sec is the results after rebuild.  This was done with a test every 5% comparable to the blue green most squiggly line from yesterday which was every 2%.  The others were 20%

 

 

 

 

The below graph is drive 2, as you can see, unchanged, and also more than twice the speed.

 

 

 

The last one below just shows the two drives right after the rebuild.

 

 

 

I have two cache SSD's and two array SSD's.  All EVO series drives.   The array drives do hit the 4X temp range during parity checks routinely but I thought SSD's were stable a good bit higher.  They are mounted in pairs pretty close together in the case but good airspace around the pair of drives.   

 

So, obviously Q1 do you think this drive is failing?   Q2 Would you replace it?  Q3 Do you agree temps in the mid 40's is acceptable for a parity check?

 

[JorgeB]

At least now it's steady, but yeah, looks like the SSD is the problem, if it's a reason to replace it now depends for what you used it, it might not make much difference.

 

As for temps, below 40 would be better, but IMO up to 45C is acceptable.

[TODDLT]

On 10/8/2018 at 4:25 AM, johnnie.black said:

At least now it's steady, but yeah, looks like the SSD is the problem, if it's a reason to replace it now depends for what you used it, it might not make much difference.

 

As for temps, below 40 would be better, but IMO up to 45C is acceptable.

I went ahead and replaced it.  No explanation for it and for a drive to now consistently run at half speed just doesn't sit well.  Speeds all back up to normal.

 

during a parity check or re-build, it bounces up between 40 and 45.  Where they are mounted in the case I can't really get a fan right on them but I think I can get one generally over the area they sit.  That's next on the list.

 

Thanks for the advice.

[UhClem]

To paraphrase The Joker, "This SSD needs an enema." (It's "constipated".)

(I've got a 850 EVO (500GB) that is also exhibiting performance degradation; but, note, this is a phenomenon common to all SSDs.) What I/we need is a very low-level format, restoring the drive to a state as close as possible to fresh from the factory.

 

Cursory research indicates that hdparm can do this, using the --trim-sector-ranges option. I have not tried this yet myself, so please excuse me for "Do as I say, not as I do." but at least it's a pointer in (I hope) the right direction.

 

RSVP with your results. tnx

 

--UhClem