Jump to content
Skrumpy

Deterministic SSD Recommendations

12 posts in this topic Last Reply

Recommended Posts

Looking for an SSD compatible with my LSI SAS 9305-24i. I have seen dozens of responses that the Samsung PRO/EVO line of SSDs does not support this and is therefor incompatible with these controllers, but I haven't seen a recommendation.

 

Does anyone have a list of drives that would work? I'd rather not downgrade my controller and get new cables, etc if I can just get a new drive instead.

 

Do the Samsung Enterprise SSDs support this? I didn't see anything specific in the specs referring to this.

Share this post


Link to post
1 hour ago, ptr727 said:

I tested EVO 840, Pro 850 and Pro 860. Pro 860 works with LSI and TRIM.

See: https://blog.insanegenius.com/2020/01/10/unraid-repeat-parity-errors-on-reboot/

 

 

I have an LSI 9305-16i SAS card and have nothing but issues when my Samsung 860 Pro undergoes TRIM. I read your post, I did not get it from The Art of Server, but the comparative card (the 9305-24i) works OOB according to unRAID comparability list.

 

Shortly after the TRIM is complete (within ~10m) several errors occur such as below with the SSD drive (formatted as XFS, but BTRFS denotes the docker/vm fs).

 

So are you saying that this card should work without issue? I have seen other posts discussing these LSI cards and Samsung drives (including the PRO). I have not had any issues with the SSD today since moving it to the mobo controller (as apparently is the default suggestion). The speeds from controller card (green) vs onboard (blue) do not get anywhere close, so I'd rather go with the controller card.

 

What do you suggest? Buy a card from Art of Server flashed to IT mode? I have SAS 8643 cables laying around and it only looks like he has 3 compatible cards. I really just want the best card I can get and to be able to plug it in and have it work.

 

image.png.34e33c1dd5aebe55bbfe0c1cca4a9b1c.png

 

image.png.0fe07f24985e28718c791bd64a69ac9f.png

 

Share this post


Link to post

I use LSI-9340-8i, SM SAS3 backplane, and 4 x 860 Pro's in BTRFS cache in two Unraid servers.

I won't call it a recommendation, but I've not seen errors with this hardware combo.

Share this post


Link to post
1 hour ago, ptr727 said:

I use LSI-9340-8i, SM SAS3 backplane, and 4 x 860 Pro's in BTRFS cache in two Unraid servers.

I won't call it a recommendation, but I've not seen errors with this hardware combo.

Interesting, I think that's good enough of an endorsement for me. Grabbing one from Art on Ebay.

 

Backplane may be a bit much for me, only running 6 drives right now, but room to grow I guess.

 

Appreciate the help!

 

Share this post


Link to post

I have an array of 4x 1.92TB Samsung Enterprise SSDs (PM863a). See hdparm output below. They have DZAT. Note that as they are in the array they do not trim, I rely on the fact that I have minimal drive writes and rely on the garbage collection. No parity errors, running for 4 months now. Maybe one day I'll play with zfs on unraid, but that's for another time long in the future.

 

/dev/sdb:

ATA device, with non-removable media
        Model Number:       SAMSUNG MZ7LM1T9HMJP-00005              
        Serial Number:      <redacted>    
        Firmware Revision:  GXT5404Q
        Transport:          Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
        Used: unknown (minor revision code 0x0039) 
        Supported: 9 8 7 6 5 
        Likely used: 9
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:    16514064
        LBA    user addressable sectors:   268435455
        LBA48  user addressable sectors:  3750748848
        Logical  Sector size:                   512 bytes
        Physical Sector size:                   512 bytes
        Logical Sector-0 offset:                  0 bytes
        device size with M = 1024*1024:     1831420 MBytes
        device size with M = 1000*1000:     1920383 MBytes (1920 GB)
        cache/buffer size  = unknown
        Form Factor: 2.5 inch
        Nominal Media Rotation Rate: Solid State Device
Capabilities:
        LBA, IORDY(can be disabled)
        Queue depth: 32
        Standby timer values: spec'd by Standard, no device specific minimum
        R/W multiple sector transfer: Max = 16  Current = 16
        Advanced power management level: disabled
        DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6 
             Cycle time: min=120ns recommended=120ns
        PIO: pio0 pio1 pio2 pio3 pio4 
             Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
        Enabled Supported:
           *    SMART feature set
                Security Mode feature set
           *    Power Management feature set
           *    Write cache
           *    Look-ahead
           *    Host Protected Area feature set
           *    WRITE_BUFFER command
           *    READ_BUFFER command
           *    NOP cmd
           *    DOWNLOAD_MICROCODE
                Advanced Power Management feature set
                Power-Up In Standby feature set
                SET_MAX security extension
           *    48-bit Address feature set
           *    Device Configuration Overlay feature set
           *    Mandatory FLUSH_CACHE
           *    FLUSH_CACHE_EXT
           *    SMART error logging
           *    SMART self-test
           *    General Purpose Logging feature set
           *    WRITE_{DMA|MULTIPLE}_FUA_EXT
           *    64-bit World wide name
                Write-Read-Verify feature set
           *    WRITE_UNCORRECTABLE_EXT command
           *    {READ,WRITE}_DMA_EXT_GPL commands
           *    Segmented DOWNLOAD_MICROCODE
                unknown 119[7]
           *    Gen1 signaling speed (1.5Gb/s)
           *    Gen2 signaling speed (3.0Gb/s)
           *    Gen3 signaling speed (6.0Gb/s)
           *    Native Command Queueing (NCQ)
           *    Phy event counters
           *    READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
           *    DMA Setup Auto-Activate optimization
           *    Asynchronous notification (eg. media change)
           *    Software settings preservation
           *    SMART Command Transport (SCT) feature set
           *    SCT Write Same (AC2)
           *    SCT Error Recovery Control (AC3)
           *    SCT Features Control (AC4)
           *    SCT Data Tables (AC5)
           *    SANITIZE_ANTIFREEZE_LOCK_EXT command
           *    SANITIZE feature set
           *    CRYPTO_SCRAMBLE_EXT command
           *    BLOCK_ERASE_EXT command
           *    reserved 69[4]
           *    DOWNLOAD MICROCODE DMA command
           *    SET MAX SETPASSWORD/UNLOCK DMA commands
           *    WRITE BUFFER DMA command
           *    READ BUFFER DMA command
           *    Data Set Management TRIM supported (limit 8 blocks)
           *    Deterministic read ZEROs after TRIM

#Remaining hdparm notes deleted. See last two bullet points above for DZAT confirmation.

 

Share this post


Link to post
On 5/7/2020 at 2:34 AM, mishmash- said:

I have an array of 4x 1.92TB Samsung Enterprise SSDs (PM863a). See hdparm output below. They have DZAT. Note that as they are in the array they do not trim, I rely on the fact that I have minimal drive writes and rely on the garbage collection. No parity errors, running for 4 months now. Maybe one day I'll play with zfs on unraid, but that's for another time long in the future.

Thank you, I assumed as much. Next build will be all SSD!

 

On 5/5/2020 at 10:39 PM, ptr727 said:

I use LSI-9340-8i, SM SAS3 backplane, and 4 x 860 Pro's in BTRFS cache in two Unraid servers.

I won't call it a recommendation, but I've not seen errors with this hardware combo.

 

Much appreciated for the link and suggestion. Art of Server came through and the LSI 9340-8i is working as described. Same benchmarks as the 9305-16i and just did a couple of test TRIM without error after adding in another 860 pro for a cache pool! I wish it had a couple of more ports but I'll worry about expansion later. Can't complain at half the price and it actually works. Thanks again!

 

image.png.e21e82cf6b6a2c05e8820d0ca6e6e47b.png

 

The compatibility wiki really should be updated to inform which boards do not support TRIM.

Share this post


Link to post

Note that partly due to legacy issues from upgrading from HDD to SSD array, I still use a cache drive SSD in BTRFS raid1. But I think I actually like having a cache SSD with an array SSD, as the cache can be trimmed, and is constantly seeing tiny writes etc. I might upgrade it to an NVME for next time. In reality though I think it does not matter at all on having cache+array ssd or just full ssd array no cache.

Share this post


Link to post
6 hours ago, mishmash- said:

Note that partly due to legacy issues from upgrading from HDD to SSD array, I still use a cache drive SSD in BTRFS raid1. But I think I actually like having a cache SSD with an array SSD, as the cache can be trimmed, and is constantly seeing tiny writes etc. I might upgrade it to an NVME for next time. In reality though I think it does not matter at all on having cache+array ssd or just full ssd array no cache.

How are your SSD's in the array? My understanding is that SSD's are not supported in the array, something to do with how parity calculations are done.

Share this post


Link to post
On 5/10/2020 at 6:02 PM, ptr727 said:

How are your SSD's in the array? My understanding is that SSD's are not supported in the array, something to do with how parity calculations are done.

Disclaimer: I'm certainly not an expert on this.

 

Support is experimental. TRIM is not available when using SSDs in the array. My basic understanding is the fact that the TRIM command is passed to the SSD controller, and there are different types of methods for performing TRIM, which will invalidate parity. It's speculated that if it ever were implemented, then the TRIM method would be DRZAT (deterministic read zeroes after TRIM), as this would in theory not break parity....or something. DRZAT is also needed for some HBA cards when using them with SSDs in cache and TRIM.

 

So if we don't have TRIM the other thing we rely on is SSD internal garbage collection, overprovisioning and general use case.

  • Garbage collection happens in the background by the controller of the SSD, and modern garbage collection methods (apparently) are quite efficient. For the most part, this does not seem to break parity either - although @johnnie.black reported one or two SSDs that were causing parity errors.
  • Overprovisioning: there's a good few posts on overprovisioning consumer drives using blkdiscard command on this forum (advanced users) or enterprise SSDs can be used, which have higher overprovision. I got a good deal on enterprise SSDs so I went with those. My original plan was to overprovision some WD Blues to enterprise levels, as these still seem to have DRZAT according to a WD Blue forum post.
  • General use case: garbage collection and TRIM requirement is a result of file deletion, so I try and minimise my writes to the drive by keeping the array mainly read data only, and having appdata still on a separate btrfs raid 1 ssd cache. It is a bit of an odd arrangement, having SSD cache and SSD array,  but putting in small consumer drives that can be TRIMmed that take most of the writes seems to work OK in this scenario.

So far uptime is about 70 days, parity check once a week with no errors so far, fingers crossed. Drives are at about 70% full. Parity check speed on a 6TB (3x2TB+parity) array is about 410MB/s average. Other R/W operations are a lot faster, presumably because it's not a constant 1 hour slog on the system.

Share this post


Link to post
36 minutes ago, mishmash- said:

although @johnnie.black reported one or two SSDs that were causing parity errors.

From the may different models I used so far there's only one that consistently had a couple of sync errors after a power cycle, it was the old Kingston V300.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.