v6 on Atom D525

[garycase]

I've seen a few posts r.e. updating SuperMicro Atom D525 boards to v6, but haven't been able to find the one I was looking for r.e. tweaks to get the parity checks to match v5 performance levels.

 

I decided to upgrade my secondary backup server, a SuperMicro D525-based server, to v6.2.4 (from v5), and it works very nicely EXCEPT parity checks are nearly twice as long as before -- averaging just over 60MB/s compared to over 120 on v5.

 

If you're running one of these and have "tweaked" it to run at v5 levels, I'd appreciate some insight into what you've done.

 

FWIW I've set the disk tunables to match what I had with v5 and disabled display updates ... but these aren't making any appreciable difference.    System has all 3TB WD Reds.

 

[RobJ]

The problem may be the tunables, if you've tweaked them for v5.  Try setting them to default.  Large adjustment to them for v5 will cause a major performance hit in v6, because the new tunable has to be close to one of the older ones.  Here's one post about it (adapt the values though).

[garycase]

I started with the defaults -- then tried using the values I'd had set in v5 -- and nothing makes any significant difference.

 

I recall seeing somebody with the same motherboard who had spent a good bit of time "fiddling" and eventually found a combination of settings that had resulted in near-v5 parity check speeds, but can't remember who that was, and a search of the forum hasn't resulted in any luck, so I was hoping he/she would see this thread and reply.

 

Don't have much time this week to mess with it -- but will probably spend some time next week to see if I can get it to work better.  Not changing the file formats anytime soon so I can revert to v5 if I choose to -- not sure I will, but it IS frustrating to see parity checks running at ~65MB/s when I know they CAN run at ~ 125MB/s 

 

[garycase]

Also just did a copy of a large file from the array to my Windows box and noticed the copy speeds have also deteriorated a LOT.  Averaged ~ 58MB/s, which seemed a lot slower than I remembered.

 

Reset to v5 and did the same copy -- averaged over 105MB/s.    Started a parity check on v5 and it's running ~ 124MB/s at the 20% point.

 

I suspect I'll just leave it on v5 for now unless somebody with the exact same board chimes in with the settings they've found to eliminate this reduction in performance.

 

[JorgeB]

For the low reed speed see if this FAQ entry helps.

 

For parity check try these tunables, they are not universal but in my experience work well with most hardware:

 

nr_requests: 128 (default works well for most controllers, use 8 if the server has either a SASLP or SAS2LP)
md_num_stripes: 4096
md_sync_window: 2048
md_sync_thresh: 2000

 

 

[landS]

I've tweaked 'acceptable'  performance out of this Gary.  I'm away from the house ATM,  but when I return I'll share specifics.    I even have dual parity.  The cache disc is a SSD.  All disks are 4 TB,  identical models.  Minimal plugins (cache directory is one  of them).  I had to run the disk tunables script as my v5 and default tunables sacked.    I also had to alter some settings on the WINDOWS  machine.  Parity speed  speeds are much faster.  Writes are around 100 using Windows,  and around 60? using Tera copy.

[landS]

Check the 'low power server'  link in my signature

[landS]

Parity 1 & 2: XFS  4 TB T4000DM000-1F2168

3 Data Drives: XFS  4 TB T4000DM000-1F2168

Cache: Samsung_SSD_850_PRO_256GB

MOBO: SuperMicro X7SPA-HF-D525

RAM: Dual Micron Memory - MEM-DR320L-CL02-SO13 - 2GB DDR3-1333

 

Plugins

Dynamix Cache Dirs

Dynamix Active Streams

Dynamix System Info

Dynamix File Integrity

IPMI Support

Open Files

Recycle Bin

Preclear Disks

Shell In a Box / Command Line Tool

Nerd Tools (for Screen)

 

Tuneables script

Poll Attributes 1800

Enable NCQ NO

Nr Request 128

Md Num Stripes 1424

Md sync window 520

Md Sync thresh 192

Md Write Method Auto

 

Last parity Check

Sun 01 Jan 2017 04:29:58 PM CST (21 days ago),

Duration: 15 hours, 59 minutes, 57 seconds. Average speed: 69.5 MB/sec

 

Yup... that's right... the parity check speeds fully tweaked are simply not as fast as v5.  For me, the added features of v6, including dual parity, are too great a benefit compared to the trade-off of 1 day per month where the check now takes 16 hours rather than 10.

 

I can eek out a little faster parity check by disabling 'most' of my plugins above... but it is not significant.

 

*Something has changed between version 6.2 and 6.2.4

*The parity check has gone from well over 71 (I believe it was ~100) to 69.5

*Note that the location (heated master bedroom closet, near the ceiling) is significantly warmer during the winter.

 

Write speed performance Boost

Max copy speed (pre-tweak) is 57 MB/s during idle & 42 MB/s during a parity check AFTER running the Disk Tuneables script

 

The following was all done on my Windows 10 machine.

When I installed the new Elementary OS release, it needed 'similiar' smb tweaks.  I have missplaced the link I followed for that.

Regedit / (DWORD value) systemCurrentControlSet/Services/lanmanworkstation/parameters/

Create the DWORD DisableBandwidthThrottling and set to 1

gpedit.msc / Computer Configuration/Administrative Templates/Network/QoS Packet Scheduler/Limit reservable bandwidth.  Edit, Enable, change 80 to 0

Settings/Updates&Security/Advanced Options/Choose how updates are delivered/Turn OFF deliveries of updates to other computers

CMD as administrator (default is 'normal')/ Netsh interface tcp set global autotuning=disabled

CMD as administrator (default is 'enabled')/ Netsh interface tcp set global rss=disabled

Programs&Features/ uncheck Remote Differential Compression

Ethernet Adapter / uncheck IPV6

Ethernet Adapter Properties / Set Speed & Duplex from Autonegotiate to 1.0 GBps Full Duplex    (note that I have a managed switch)

Disable Unused Network Adapters

 

And the results after rebooting the Windows 10 machine?

File copies DURING a dual parity check using TeraCopy are now at 45, and 64 at all other times.

File copies DURING a dual parity check using Windows Explorer are now at 78, and 98 at all other times.

 

[garycase]

...

Last parity Check

Sun 01 Jan 2017 04:29:58 PM CST (21 days ago),

Duration: 15 hours, 59 minutes, 57 seconds. Average speed: 69.5 MB/sec

 

Yup... that's right... the parity check speeds fully tweaked are simply not as fast as v5.  For me, the added features of v6, including dual parity, are too great a benefit compared to the trade-off of 1 day per month where the check now takes 16 hours rather than 10.

 

Ouch!  Those are consistent with my results -- for some reason I thought I'd remembered you having found a way to get the speed back up to near-v5 levels (actually it sounds like that might have been the case with earlier v6 versions -- I might have to try v6.1.9 and see what it does).    My D525 system has all 3TB WD Reds, and the parity checks went from 8 hrs (v5) to 13 hrs.  I agree it's not a big deal, but it's nevertheless frustrating to know they COULD be so much faster.

 

For now, my system is simply back to v5 ... and I'll probably just leave it that way until I have more time to "play" around a bit with the settings.  Wife just got out of the hospital after major surgery, so I'm tied up as a full-time "nursemaid" for the next few weeks.

 

 

[landS]

Earlier  v6 post tunables shaved about 2 hours off of the 15.5 hour check.    It took me a LONG time to tweak WRITE performance back up....  That one did bother me a great deal.. . But now that I'm back up to v5 write performance I am happy.    Note that the READ is a bit limited on speed during the parity check.

[garycase]

I NEVER use the array during parity checks, so read/write speeds during that time aren't of any concern.  For that matter, the slower parity checks really don't matter, other than knowing that they COULD (and IMHO SHOULD) be so much faster.

 

But it definitely bugs me that the transfer speeds are so much slower.    When I get a chance I'll try Johnnie's suggestion r.e. limiting the SMB level  (I'd try that now if I hadn't already reverted back to v5).    If that resolves the transfer speed issue, I'll probably just stay with v6 and not worry about the parity check speed -- I DO like the GUI improvements.

 

But the reality is the version doesn't make much difference for this particular server -- it's purely a NAS, has very few plugins (CacheDirs and PowerDown and UPS support), and is rock solid as is.    I only did the v6 upgrade so all of my servers would be running the same version.

 

 

 

[garycase]

landS => By the way, your "low power server" thread [ http://lime-technology.com/forum/index.php?topic=27955.0 ] was exactly what I had remembered seeing and was referring to in my initial post in this thread 

 

I will try the specific settings you're using ... and the tweaks Johnnie noted in his post above ... sometime in the next week or so when I get some time to "fiddle" again with v6.

 

[landS]

Yup,  something in the newer SMB broke transfer speeds.  Did you see my Windows tweaks that allow you to use the newer SMB while reclaiming the speeds?  Works great...  And actually helps with other Windows network speeds (including speakeasy / Internet speeds)

[Smitty2k1]

I was never able to get write speeds to acceptable levels in all of my time with v6 on the D525. Even with an SSD cache write speeds were still around 40MB/s.

 

But if you guys get this all figured out and anyone wants another, I've got the super micro board with 4GB RAM I'd unload for super cheap.

[garycase]

Okay, curiosity got to me, so I switched it back to the clean v6.2.4 install and tried the settings suggested by Johnnie (SMB limit to v2 and the disk tunables); and also tried the same tunables landS is using.

 

No appreciable gains in either parity check speed or transfer speeds from the array.    Spent about an hour testing various combinations -- but I'm simply not happy with the results, so I'm just going to switch back to v5 and this system shall remain there.

 

It's a ROCK SOLID system and works just fine with v5, so it'll just stay there.

 

BTW, I did NOT change any settings on my Windows 10 system -- it works perfectly with both of my v6.2.4 systems (excellent data transfer speeds, etc.), so I am confident that's not an issue.

 

It was instructive to try v6.2.4, but I'll simply stay with v5.0.6 for it's far better performance.    Coupled with UnMenu, it does absolutely everything I want it to for its basic NAS functionality.

 

[landS]

Enjoy v5 Gary

[garycase]

Enjoy v5 Gary

 

:)

 

This system is used purely as a NAS -- in reality, there's NO advantage to moving it to v6 and a nice speed advantage in staying with v5.    I almost never look at the Web GUI anyway except to do a parity check.    The machine's turned on by a script file;  some backups are done; and then it's turned off.    When I want to access a file from the backups or save something to it, I double-click an icon that turns on the system; then use Windows Explorer to read or write the files.  No GUI access involved, so from the user perspective it's entirely irrelevant whether it's running v5, v6, etc.

 

 

[RobJ]

This system is used purely as a NAS -- in reality, there's NO advantage to moving it to v6 and a nice speed advantage in staying with v5.    I almost never look at the Web GUI anyway except to do a parity check.    The machine's turned on by a script file;  some backups are done; and then it's turned off.    When I want to access a file from the backups or save something to it, I double-click an icon that turns on the system; then use Windows Explorer to read or write the files.  No GUI access involved, so from the user perspective it's entirely irrelevant whether it's running v5, v6, etc.

 

Respectfully Gary, on principle I have to disagree with the highlighted statement above.  I fear a v5 user will see your comments, from a highly respected veteran, and use them as an excuse to stay with v5.  That user may keep the server on much more, and may not be as vigilant, and may have relatives with risky computer behaviors.  I want to remind users that in Gary's usage of that v5 server, he greatly limits the access windows, the windows of vulnerability.  While what he's doing may be OK for a knowledgeable and advanced user, who understands the risks, it can never be a recommended course of action.

 

All v5 releases are insecure.  For that matter, v6.0, v6.1, and even v6.2 releases are not as secure as the latest unRAID release (v6.3-RC).  For security reasons alone, it's not wise to stay with v5, but for even more reasons, please see this message.

[ashman70]

What does 'insecure' mean? For a home network user not exposing their unRAID server to the internet and only using it as a NAS, 'insecure' doesn't really have any ramifications. Once you start opening ports and exposing your unRAID server to the Internet, that changes entirely. I think the way Gary has framed what he is saying, around 'used purely as a NAS' makes sense, there is no interest in using dockers, or virtual machines or plugins, if there is, upgrade to v6, otherwise v5 is perfectly fine for 'as a NAS' use.

[landS]

Devils advocate:  just about every household that has kids with an Xbox playing online likely has ports punched wide open  through the router... 

[Squid]

Devils advocate:  just about every household that has kids with an Xbox playing online likely has ports punched wide open  through the router...

Its actually far worse than that with the IoT devices out now with everyone jockeying for market share and not worrying at all about the WiFi security inside.

[garycase]

Rob => Agree r.e. security.    My comment was focused purely on the NAS functionality => v6 provides a nicer GUI, built-in UPS support, etc. -- and of course supports a more modern file system;  but none of these really make it more reliable and "rock solid" than v5 does.    I do NOT understand why 6.2.4 is so much slower on parity checks and file transfers -- but it is what it is, so for what I use the system for, I'll take the higher performance of v5.

 

Agree, however, that if you're exposing your systems to the internet, it's best to use the most current version so you have a reasonably up-to-date set of security protections.

 

 

[JonathanM]

Agree, however, that if you're exposing your systems to the internet, it's best to use the most current version so you have a reasonably up-to-date set of security protections.

You should amend that to say if you are exposing your systems to ANY internet connected devices, you should have the best security reasonably available to you. It only takes one compromised system behind the firewall to serve as a gateway.

[garycase]

An update after another year -- I finally decided I simply didn't care about the speed issue, so I updated this system to 6.3.5.   As expected, parity checks now take a bit over 16 hours (nearly double what they do with v5), and read/write speeds are appreciably slower => but the reality is I still just use this system as a "pure NAS" and the speed simply doesn't matter.

 

I didn't do the upgrade for improved security (although that's a nice extra benefit) -- I simply did it so all of my UnRAID systems are running the same version.

 

r.e. last year's comments about needing the extra security if you have any exposure to the internet -- I agree; but this system has no Dockers, no VM's, and only one plugin (CacheDirs) -- and I suspect most folks with D525-based systems have similar setups.   If you want to do more, you really need more "horsepower" than the dated D525 provides.   But for basic NAS duty it's hard to beat this system with it's 19 watt idle power consumption.

 

It conceptually still bugs me that v6 is SO much slower running a parity check ... there's simply no technical reason why that should be the case => but I'll just live with it.

 

landS =>  Have you by any chance discovered any further "tweaks" that help with v6 disk speeds?

 

 

[dalben]

On 1/22/2017 at 5:05 PM, johnnie.black said:

For the low reed speed see if this FAQ entry helps.

 

For parity check try these tunables, they are not universal but in my experience work well with most hardware:

nr_requests: 128 (default works well for most controllers, use 8 if the server has either a SASLP or SAS2LP)
md_num_stripes: 4096
md_sync_window: 2048
md_sync_thresh: 2000

 

 

jb, can you give some further detail on

nr_requests: 128 (default works well for most controllers, use 8 if the server has either a SASLP or SAS2LP)

I've recently installed an LSI SAS9211-8i card to drive my SATA disks.  I'm not sure whether your SASLP/SAS2LP comment relates to my config or not.