Recently upgraded with a DAS w/ SAS2-826EL1 backplane & LSI 9207-8e HBA to allow for TRIM support on SSDs.  I currently have an 850 EVO 500gb (sdb) as my cache drive on motherboard SATA port which is working 100% & have an 850 EVO 250gb (sdl) on the DAS but have not been able to get TRIM working.  I formatted the 250gb ext4 after doing some reading under the impression that this filesystem allows for TRIM, but I would like to move my cache drive to 6gb/s HBA as opposed to the 3gb/s my motherboard allows.  The Broadcom compatibility guide lists both drives as compatible with 9207-8e, so is there any guidance as to what I may be doing wrong or missing?

Edited September 28, 20187 yr by therapist

There are been some reports of trimm not working on the 9207 with latest Unraid releases, that likely include a newer LSI driver.

4 minutes ago, johnnie.black said:

There are been some reports of trimm not working on the 9207 with latest Unraid releases, that likely include a newer LSI driver.

Formatting an SSD to ext4 and using under Unassigned Devices would be a viable alternative?

 

Can a cache disk be reformatted to ext4 if so?

Just now, therapist said:

Formatting an SSD to ext4 and using under Unassigned Devices would be a viable alternative?

Same problem if still on the 9207.

 

1 minute ago, therapist said:

Can a cache disk be reformatted to ext4 if so?

No, xfs or btrfs (or the not recommended reiser fs)

I know this is hard to extrapolate -- but what are the real world implications for no TRIM on a "consumer grade" SSD?

The drive hosts VM images will garbage collection suffice?

Depends on the use, also on the SSD and the amount of spare flash it comes with, you can also overprovision a little if you want to void write performance degradation.

So I purchased a Samsung 840 PRO SSD as it is the only drive I can confirm has DRAT

Formatted BTRFS in UD -- mounted & tried FSTRIM -- got "remote I/O failed" error

Formatted ext4 in UD -- mounted & trimmed -- BINGO!

 

so maybe fstrim doesnt like BTRFS on HBAs....lets try xfs (this device is destined to be a cache disk after all)

Formatted XFS in UD -- mounted & tried to TRIM -- "remote I/O failed"

 

WTF?

 

@limetech is there something with unraid that prevents HBA trim of BTRFS/XFS formatted devices?

Any thought on allowing ext4 formatted cache disks?

 

Any ideas @johnnie.black?

 

It would be really nice to be able to utilize my higher bandwidth drives to their max, rather then gimp them with onboard 3Gb/s SATA

45 minutes ago, therapist said:

is there something with unraid that prevents HBA trim of BTRFS/XFS formatted devices?

TRIM isn't supported for parity-protected array.  Should work fine for cache and/or UD plugin.  Posting diagnostics.zip might yield more info.

  

diagnostics attached

trim does work...my current cache ssd works fine on the  X8DTI-F SATA connection and I was able to get the new 840 PRO to trim on the 9207-8e connected DAS, but only when the disk was ext4 formatted?

 

ocho-diagnostics-20181004-1717.zip

1 hour ago, therapist said:

X8DTI-F SATA connection and I was able to get the new 840 PRO to trim on the 9207-8e connected DAS, but only when the disk was ext4 formatted?

There have been some reports of trim not working any longer on the LSI 9207, IIRC since around v6.5, possibly same change on the LSI driver, but I don't know if it's a general issue, I do have one and trim worked fine before, but not been using it with SSDs for while, I'll try to make some tests when I can.

Same problem here, my 840 Pro worked fine, but I now see the error:

 

FITRIM ioctl failed: Remote I/O error

 

Did some testing using my 9207, trim works up to Unraid v6.3.5, looking at the LSI driver included with each Unraid release:

 

v6.3.5 - 13.100.00.00 - trim works

v6.4.1 - 15.100.00.00 - trim doesn't work anymore

v6.6.1 - 25.100.00.00 - trim still doesn't work

 

Tried looking at the driver release notes and didn't find any trim mentions, if it was a driver bug I would assume it would be fixed since 15.100.00.00, possibly there were some issues and trim wasn't working correctly so they disable it, but just speculating, in any case it looks like for the near future SAS2308 based LSI controllers, as was already the case for SAS2008, won't support trim on most (if not any) SSDs, so they should be avoided with SSDs, use the onboard SATA ports or other AHCI controller, e.g., the cheap 2 port ASMEDIA 1601 based controllers.

 

1 hour ago, johnnie.black said:

Did some testing using my 9207, trim works up to Unraid v6.3.5, looking at the LSI driver included with each Unraid release:

 

v6.3.5 - 13.100.00.00 - trim works

v6.4.1 - 15.100.00.00 - trim doesn't work anymore

v6.6.1 - 25.100.00.00 - trim still doesn't work

 

Tried looking at the driver release notes and didn't find any trim mentions, if it was a driver bug I would assume it would be fixed since 15.100.00.00, possibly there were some issues and trim wasn't working correctly so they disable it, but just speculating, in any case it looks like for the near future SAS2308 based LSI controllers, as was already the case for SAS2008, won't support trim on most (if not any) SSDs, so they should be avoided with SSDs, use the onboard SATA ports or other AHCI controller, e.g., the cheap 2 port ASMEDIA 1601 based controllers.

 

Thank you for going through this!

At least I know my hardware may not be at fault...

 

Should this be submitted as a bug report?

Should this be submitted as a bug report?

Do you mean Unraid bug report? The problem is the LSI driver so you'd need to contact them or whoever maintains it.

 

 

I suppose youre right...

 

@limetech is there any way that the end used can replace the LSI driver similar to:

https://thevirtualist.org/replacing-driver-for-megaraid-sas-9361-8i-on-esxi-6/

 

It would be far easier to do this than keep buying more hardware to work around a software issue

On 10/5/2018 at 7:13 AM, johnnie.black said:

Did some testing using my 9207, trim works up to Unraid v6.3.5, looking at the LSI driver included with each Unraid release:

 

v6.3.5 - 13.100.00.00 - trim works

v6.4.1 - 15.100.00.00 - trim doesn't work anymore

v6.6.1 - 25.100.00.00 - trim still doesn't work

 

Any ideas on why an ext4 formatted drive will still TRIM properly w/ current drivers?

15 hours ago, therapist said:

Any ideas on why an ext4 formatted drive will still TRIM properly w/ current drivers?

No clue.  Has a google search of this issue turned up anything?

On 10/7/2018 at 12:41 PM, limetech said:

No clue.  Has a google search of this issue turned up anything?

I find a few references to SSDs working with TRIM on UNRAID, but nothing recently which would coincide with LSI driver updates.  I wonder how many people out there had this working & just don't realize they are no longer supported...

 

In the mean time I've figured out that I can manually TRIM my HBA bound SSDs by stopping array, removing from chassis & hot plugging to mobo.  Then mount with UD, fstrim & return to normally scheduled programming.

 

The LSI driver is baked in to UNRAID, so is there any advice on where to bring up this issue with the keepers of that code?

3 minutes ago, therapist said:

I find a few references to SSDs working with TRIM on UNRAID, but nothing recently which would coincide with LSI driver updates.  I wonder how many people out there had this working & just don't realize they are no longer supported...

 

In the mean time I've figured out that I can manually TRIM my HBA bound SSDs by stopping array, removing from chassis & hot plugging to mobo.  Then mount with UD, fstrim & return to normally scheduled programming.

 

The LSI driver is baked in to UNRAID, so is there any advice on where to bring up this issue with the keepers of that code?

I'm in the same boat.  I've been playing with the idea of upgrading my HBA to a 9300-8i, but I'm not entirely sure it will resolve the trim issue.  And before it is recommended again, I know that I can connect my SSDs to the motherboard to regain the trim capability, but with having my hard drives located in another chassis, I'd rather not.  

 

I haven't had a lot of time to dissect the drivers in this latest update, but I know one thing... Unraid is Linux.... and I know Linux.... (just had a weird flash back)

 

1 hour ago, debit lagos said:

I've been playing with the idea of upgrading my HBA to a 9300-8i, but I'm not entirely sure it will resolve the trim issue.

There are reports that the issue remains even with the newer SAS3 models, like the 9300i, 9305, etc.

34 minutes ago, johnnie.black said:

There are reports that the issue remains even with the newer SAS3 models, like the 9300i, 9305, etc.

Thanks for confirming that.  I've spent the past 30 minutes or so digging through the mpt3sas kernel module code, comparing it to previous versions.  Unfortunately I'm down a monitor due to my little sitting with me, watching Peppa Pig... oh the sacrifices...

 

 

Edited October 13, 20187 yr by debit lagos

Not to go overboard on exploring this situation, I'm working to gather all the same pieces in my current setup, with the exception of all the array drives, and just a few SSDs.  Can't afford to screw up my main system.

 

Furthermore, there seems to be several different things going on.  One being, the migration of mpt2sas and mpt3sas.  With those two converged, on the surface, nothing should have changed.  One thing I came across was a linux tool called sg_unmap.  It communicates vis SCSI rather ATA.  I want to see if the issue is communication commands, but i'm unsure how to sideload packages on unraid.  

 

Lastly, the package in which the command/program fstrim resides, there is an update to that package 2.32 > 2.33  that mentions a few things about FITRIM ioctl failed.  Does anyone know how to update this package (util-linux)?  Package was released on the 19th of October (https://mirrors.edge.kernel.org/pub/linux/utils/util-linux/)

 

The journey continues...

Thanks for keeping the light on with this

 

I still cant figure why I was able to TRIM ext4 formatted disks

On ‎10‎/‎21‎/‎2018 at 10:48 AM, debit lagos said:

Not to go overboard on exploring this situation, I'm working to gather all the same pieces in my current setup, with the exception of all the array drives, and just a few SSDs.  Can't afford to screw up my main system.

 

Furthermore, there seems to be several different things going on.  One being, the migration of mpt2sas and mpt3sas.  With those two converged, on the surface, nothing should have changed.  One thing I came across was a linux tool called sg_unmap.  It communicates vis SCSI rather ATA.  I want to see if the issue is communication commands, but i'm unsure how to sideload packages on unraid.  

 

Lastly, the package in which the command/program fstrim resides, there is an update to that package 2.32 > 2.33  that mentions a few things about FITRIM ioctl failed.  Does anyone know how to update this package (util-linux)?  Package was released on the 19th of October (https://mirrors.edge.kernel.org/pub/linux/utils/util-linux/)

 

The journey continues...

 

Does not appear that @limetech is updating the util-linux or mpt3sas packages in the 6.6.0 RCs

mpt3sas now up to 27.100.00.00.

37 minutes ago, therapist said:

Does not appear that @limetech is updating the util-linux or mpt3sas packages in the 6.6.0 RCs

mpt3sas now up to 27.100.00.00.

 

There are no longer any 6.6.0-rc's since 6.6 has been released as 'stable'.

re: util-linux - we avoid large package updates in stable point releases, unless required for security patch, or other crucial bug fix.  The next update of that package will be in 6.7.

re: mpt3sas: generally anything other than 'bug fixes' are not backported from linux RC's.  We are also updating to 4.19 kernel in Unraid 6.7.  Not sure if the mptsas driver is updated there or not.