therapist Posted September 28, 2018 Share Posted September 28, 2018 (edited) Recently upgraded with a DAS w/ SAS2-826EL1 backplane & LSI 9207-8e HBA to allow for TRIM support on SSDs. I currently have an 850 EVO 500gb (sdb) as my cache drive on motherboard SATA port which is working 100% & have an 850 EVO 250gb (sdl) on the DAS but have not been able to get TRIM working. I formatted the 250gb ext4 after doing some reading under the impression that this filesystem allows for TRIM, but I would like to move my cache drive to 6gb/s HBA as opposed to the 3gb/s my motherboard allows. The Broadcom compatibility guide lists both drives as compatible with 9207-8e, so is there any guidance as to what I may be doing wrong or missing? Edited September 28, 2018 by therapist Quote Link to comment
JorgeB Posted September 28, 2018 Share Posted September 28, 2018 There are been some reports of trimm not working on the 9207 with latest Unraid releases, that likely include a newer LSI driver. Quote Link to comment
therapist Posted September 28, 2018 Author Share Posted September 28, 2018 4 minutes ago, johnnie.black said: There are been some reports of trimm not working on the 9207 with latest Unraid releases, that likely include a newer LSI driver. Formatting an SSD to ext4 and using under Unassigned Devices would be a viable alternative? Can a cache disk be reformatted to ext4 if so? Quote Link to comment
JorgeB Posted September 28, 2018 Share Posted September 28, 2018 Just now, therapist said: Formatting an SSD to ext4 and using under Unassigned Devices would be a viable alternative? Same problem if still on the 9207. 1 minute ago, therapist said: Can a cache disk be reformatted to ext4 if so? No, xfs or btrfs (or the not recommended reiser fs) Quote Link to comment
therapist Posted September 28, 2018 Author Share Posted September 28, 2018 I know this is hard to extrapolate -- but what are the real world implications for no TRIM on a "consumer grade" SSD? The drive hosts VM images will garbage collection suffice? Quote Link to comment
JorgeB Posted September 28, 2018 Share Posted September 28, 2018 Depends on the use, also on the SSD and the amount of spare flash it comes with, you can also overprovision a little if you want to void write performance degradation. Quote Link to comment
therapist Posted October 4, 2018 Author Share Posted October 4, 2018 So I purchased a Samsung 840 PRO SSD as it is the only drive I can confirm has DRAT Formatted BTRFS in UD -- mounted & tried FSTRIM -- got "remote I/O failed" error Formatted ext4 in UD -- mounted & trimmed -- BINGO! so maybe fstrim doesnt like BTRFS on HBAs....lets try xfs (this device is destined to be a cache disk after all) Formatted XFS in UD -- mounted & tried to TRIM -- "remote I/O failed" WTF? @limetech is there something with unraid that prevents HBA trim of BTRFS/XFS formatted devices? Any thought on allowing ext4 formatted cache disks? Any ideas @johnnie.black? It would be really nice to be able to utilize my higher bandwidth drives to their max, rather then gimp them with onboard 3Gb/s SATA Quote Link to comment
limetech Posted October 4, 2018 Share Posted October 4, 2018 45 minutes ago, therapist said: is there something with unraid that prevents HBA trim of BTRFS/XFS formatted devices? TRIM isn't supported for parity-protected array. Should work fine for cache and/or UD plugin. Posting diagnostics.zip might yield more info. Quote Link to comment
therapist Posted October 4, 2018 Author Share Posted October 4, 2018 diagnostics attached trim does work...my current cache ssd works fine on the X8DTI-F SATA connection and I was able to get the new 840 PRO to trim on the 9207-8e connected DAS, but only when the disk was ext4 formatted? ocho-diagnostics-20181004-1717.zip Quote Link to comment
JorgeB Posted October 4, 2018 Share Posted October 4, 2018 1 hour ago, therapist said: X8DTI-F SATA connection and I was able to get the new 840 PRO to trim on the 9207-8e connected DAS, but only when the disk was ext4 formatted? There have been some reports of trim not working any longer on the LSI 9207, IIRC since around v6.5, possibly same change on the LSI driver, but I don't know if it's a general issue, I do have one and trim worked fine before, but not been using it with SSDs for while, I'll try to make some tests when I can. 1 Quote Link to comment
markswift Posted October 5, 2018 Share Posted October 5, 2018 Same problem here, my 840 Pro worked fine, but I now see the error: FITRIM ioctl failed: Remote I/O error Quote Link to comment
JorgeB Posted October 5, 2018 Share Posted October 5, 2018 Did some testing using my 9207, trim works up to Unraid v6.3.5, looking at the LSI driver included with each Unraid release: v6.3.5 - 13.100.00.00 - trim works v6.4.1 - 15.100.00.00 - trim doesn't work anymore v6.6.1 - 25.100.00.00 - trim still doesn't work Tried looking at the driver release notes and didn't find any trim mentions, if it was a driver bug I would assume it would be fixed since 15.100.00.00, possibly there were some issues and trim wasn't working correctly so they disable it, but just speculating, in any case it looks like for the near future SAS2308 based LSI controllers, as was already the case for SAS2008, won't support trim on most (if not any) SSDs, so they should be avoided with SSDs, use the onboard SATA ports or other AHCI controller, e.g., the cheap 2 port ASMEDIA 1601 based controllers. Quote Link to comment
therapist Posted October 5, 2018 Author Share Posted October 5, 2018 1 hour ago, johnnie.black said: Did some testing using my 9207, trim works up to Unraid v6.3.5, looking at the LSI driver included with each Unraid release: v6.3.5 - 13.100.00.00 - trim works v6.4.1 - 15.100.00.00 - trim doesn't work anymore v6.6.1 - 25.100.00.00 - trim still doesn't work Tried looking at the driver release notes and didn't find any trim mentions, if it was a driver bug I would assume it would be fixed since 15.100.00.00, possibly there were some issues and trim wasn't working correctly so they disable it, but just speculating, in any case it looks like for the near future SAS2308 based LSI controllers, as was already the case for SAS2008, won't support trim on most (if not any) SSDs, so they should be avoided with SSDs, use the onboard SATA ports or other AHCI controller, e.g., the cheap 2 port ASMEDIA 1601 based controllers. Thank you for going through this! At least I know my hardware may not be at fault... Should this be submitted as a bug report? Quote Link to comment
JorgeB Posted October 5, 2018 Share Posted October 5, 2018 Should this be submitted as a bug report? Do you mean Unraid bug report? The problem is the LSI driver so you'd need to contact them or whoever maintains it. Quote Link to comment
therapist Posted October 5, 2018 Author Share Posted October 5, 2018 I suppose youre right... @limetech is there any way that the end used can replace the LSI driver similar to: https://thevirtualist.org/replacing-driver-for-megaraid-sas-9361-8i-on-esxi-6/ It would be far easier to do this than keep buying more hardware to work around a software issue Quote Link to comment
therapist Posted October 7, 2018 Author Share Posted October 7, 2018 On 10/5/2018 at 7:13 AM, johnnie.black said: Did some testing using my 9207, trim works up to Unraid v6.3.5, looking at the LSI driver included with each Unraid release: v6.3.5 - 13.100.00.00 - trim works v6.4.1 - 15.100.00.00 - trim doesn't work anymore v6.6.1 - 25.100.00.00 - trim still doesn't work Any ideas on why an ext4 formatted drive will still TRIM properly w/ current drivers? Quote Link to comment
limetech Posted October 7, 2018 Share Posted October 7, 2018 15 hours ago, therapist said: Any ideas on why an ext4 formatted drive will still TRIM properly w/ current drivers? No clue. Has a google search of this issue turned up anything? Quote Link to comment
therapist Posted October 13, 2018 Author Share Posted October 13, 2018 On 10/7/2018 at 12:41 PM, limetech said: No clue. Has a google search of this issue turned up anything? I find a few references to SSDs working with TRIM on UNRAID, but nothing recently which would coincide with LSI driver updates. I wonder how many people out there had this working & just don't realize they are no longer supported... In the mean time I've figured out that I can manually TRIM my HBA bound SSDs by stopping array, removing from chassis & hot plugging to mobo. Then mount with UD, fstrim & return to normally scheduled programming. The LSI driver is baked in to UNRAID, so is there any advice on where to bring up this issue with the keepers of that code? Quote Link to comment
debit lagos Posted October 13, 2018 Share Posted October 13, 2018 3 minutes ago, therapist said: I find a few references to SSDs working with TRIM on UNRAID, but nothing recently which would coincide with LSI driver updates. I wonder how many people out there had this working & just don't realize they are no longer supported... In the mean time I've figured out that I can manually TRIM my HBA bound SSDs by stopping array, removing from chassis & hot plugging to mobo. Then mount with UD, fstrim & return to normally scheduled programming. The LSI driver is baked in to UNRAID, so is there any advice on where to bring up this issue with the keepers of that code? I'm in the same boat. I've been playing with the idea of upgrading my HBA to a 9300-8i, but I'm not entirely sure it will resolve the trim issue. And before it is recommended again, I know that I can connect my SSDs to the motherboard to regain the trim capability, but with having my hard drives located in another chassis, I'd rather not. I haven't had a lot of time to dissect the drivers in this latest update, but I know one thing... Unraid is Linux.... and I know Linux.... (just had a weird flash back) Quote Link to comment
JorgeB Posted October 13, 2018 Share Posted October 13, 2018 1 hour ago, debit lagos said: I've been playing with the idea of upgrading my HBA to a 9300-8i, but I'm not entirely sure it will resolve the trim issue. There are reports that the issue remains even with the newer SAS3 models, like the 9300i, 9305, etc. Quote Link to comment
debit lagos Posted October 13, 2018 Share Posted October 13, 2018 (edited) 34 minutes ago, johnnie.black said: There are reports that the issue remains even with the newer SAS3 models, like the 9300i, 9305, etc. Thanks for confirming that. I've spent the past 30 minutes or so digging through the mpt3sas kernel module code, comparing it to previous versions. Unfortunately I'm down a monitor due to my little sitting with me, watching Peppa Pig... oh the sacrifices... Edited October 13, 2018 by debit lagos 1 Quote Link to comment
debit lagos Posted October 21, 2018 Share Posted October 21, 2018 Not to go overboard on exploring this situation, I'm working to gather all the same pieces in my current setup, with the exception of all the array drives, and just a few SSDs. Can't afford to screw up my main system. Furthermore, there seems to be several different things going on. One being, the migration of mpt2sas and mpt3sas. With those two converged, on the surface, nothing should have changed. One thing I came across was a linux tool called sg_unmap. It communicates vis SCSI rather ATA. I want to see if the issue is communication commands, but i'm unsure how to sideload packages on unraid. Lastly, the package in which the command/program fstrim resides, there is an update to that package 2.32 > 2.33 that mentions a few things about FITRIM ioctl failed. Does anyone know how to update this package (util-linux)? Package was released on the 19th of October (https://mirrors.edge.kernel.org/pub/linux/utils/util-linux/) The journey continues... 1 1 Quote Link to comment
therapist Posted October 22, 2018 Author Share Posted October 22, 2018 Thanks for keeping the light on with this I still cant figure why I was able to TRIM ext4 formatted disks Quote Link to comment
therapist Posted October 29, 2018 Author Share Posted October 29, 2018 On 10/21/2018 at 10:48 AM, debit lagos said: Not to go overboard on exploring this situation, I'm working to gather all the same pieces in my current setup, with the exception of all the array drives, and just a few SSDs. Can't afford to screw up my main system. Furthermore, there seems to be several different things going on. One being, the migration of mpt2sas and mpt3sas. With those two converged, on the surface, nothing should have changed. One thing I came across was a linux tool called sg_unmap. It communicates vis SCSI rather ATA. I want to see if the issue is communication commands, but i'm unsure how to sideload packages on unraid. Lastly, the package in which the command/program fstrim resides, there is an update to that package 2.32 > 2.33 that mentions a few things about FITRIM ioctl failed. Does anyone know how to update this package (util-linux)? Package was released on the 19th of October (https://mirrors.edge.kernel.org/pub/linux/utils/util-linux/) The journey continues... Does not appear that @limetech is updating the util-linux or mpt3sas packages in the 6.6.0 RCs mpt3sas now up to 27.100.00.00. Quote Link to comment
limetech Posted October 29, 2018 Share Posted October 29, 2018 37 minutes ago, therapist said: Does not appear that @limetech is updating the util-linux or mpt3sas packages in the 6.6.0 RCs mpt3sas now up to 27.100.00.00. There are no longer any 6.6.0-rc's since 6.6 has been released as 'stable'. re: util-linux - we avoid large package updates in stable point releases, unless required for security patch, or other crucial bug fix. The next update of that package will be in 6.7. re: mpt3sas: generally anything other than 'bug fixes' are not backported from linux RC's. We are also updating to 4.19 kernel in Unraid 6.7. Not sure if the mptsas driver is updated there or not. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.