Jump to content
limetech

Unraid OS version 6.7.1-rc1 available

35 posts in this topic Last Reply

Recommended Posts

This is a special release in light of the recent so-called Zombieland vulnerability revealed by Intel earlier this week.

 

Normally we don't generate -rc stable patch releases, however in the interest of maintaining our, and the Community's sanity, this release is exactly the same as 6.7.0 except for updated Intel CPU microcode and corresponding Linux kernel (4.19.43).  If 6.7.1-rc1 "fails" with something but 6.7.0 "works" then we can be fairly confident microcode/kernel to blame.  We have released this on the next branch in order to get some testing before publishing for everyone on stable.

 

Please post here in this topic, any issues you run across which are not present in 6.7.0 also - that is, issue that you think can be directly attributed to the microcode and/or kernel changes.

 

Version 6.7.1-rc1 2019-05-17

Linux kernel:

  • version: 4.19.43
  • intel-microcode: version 20190514a
  • Like 2
  • Upvote 1

Share this post


Link to post
10 minutes ago, mattgob86 said:

So if we are running AMD this won't have any change?

Unless something broke inadvertently in the kernel.

Share this post


Link to post
Posted (edited)

Thanks for being on top of this. 

Off topic: These vulnerabilities are becoming a nightmare for the performance. Last time it affected the performance and I vaguely remember reading something about that this patch also would impact the performance. Soon our powerfull unraid servers will be a slow as hell, barely able to run as a plain NAS.. 

Edited by strike
  • Like 1
  • Upvote 1

Share this post


Link to post

This isn't about unraid, more of general rant...

 

At this point, I'd rather have an option to enable and disable the patches which are focused and directed on cloud-hosted servers or servers that would be targeted and have actual secure data to protect. Why should normal consumers be forced to run performance heavy security patches? They should be set as a compliance level, like DOD.

 

General security patches are fine since they dont come with 30% perf hits.

 

Share this post


Link to post
9 minutes ago, BRiT said:

At this point, I'd rather have an option to enable and disable the patches

Well, I think I have to agree now that I think about it. I'd like to have a powerful and a secure server but it seems that I can't have both anymore.  Since unraid 6  my unraid server does so much in my home that I'd hate to lose any of it on account of performance. My data is valuable to me but not THAT valuable that I risk a 30-50% performance hit. Then I'd rather lose my data and restore from backup. Cause like all seasoned unraid users I do have backups.

Share this post


Link to post

There are several ways to disable the various mitigations, both via kernel parameters and at run-time (though this will require us to include 'debugfs' which we can do).  Google 'linux disable meltdown spectre zombieland mitigations' for a number of how-to articles.  If it gets serious enough we'll probably add debugfs support and perhaps a config page.  Having all these mitigations in place is mainly a C.Y.A. move in my opinion.

  • Like 3
  • Upvote 2

Share this post


Link to post
3 hours ago, Dazog said:

And this:

phoronix articles are good but the comments always make my eyes bleed 😱

Share this post


Link to post
5 hours ago, limetech said:

There are several ways to disable the various mitigations, both via kernel parameters and at run-time (though this will require us to include 'debugfs' which we can do).  Google 'linux disable meltdown spectre zombieland mitigations' for a number of how-to articles.  If it gets serious enough we'll probably add debugfs support and perhaps a config page.  Having all these mitigations in place is mainly a C.Y.A. move in my opinion.

I would be interested in this, right after multiple cache pools and arrays!

Share this post


Link to post

So when will someone start a class action suit against Intel. Seriously. If you purchase a processor for a certain performance / dollar value you expect it to be what you purchased? Now with the patches in regards to the list of vulnerabilities in the last couple of years there can easily be a 30% performance decrease. Which means the performance / dollar is no longer being respected.

 

Seeing that Intel charges the big price for their processors it would be about time that they pay back their customers!

 

I am a big fan of AMD but they fell behind in terms of performance. But seeing that the performance will now be at par with Intel, after patch installs, and the cost being cheaper... I think I will be going back to them. 

 

Share this post


Link to post
1 hour ago, ximian said:

I am a big fan of AMD but they fell behind in terms of performance. But seeing that the performance will now be at par with Intel, after patch installs, and the cost being cheaper... I think I will be going back to them. 

Unfortunately performance isn't the only metric. General compatibility and ease of implementation with various advanced functions like hardware pass through and such also tip on intel's side. I would say that you can eventually see progress and things get worked out, like the Ryzen and Threadripper issues, but it seems like by the time everything settles out and works well, the products are stale, and it's time to start the cycle of incompatibility and fixes again.

 

"Maybe next time it will be different" isn't a comfortable way to approach tech. I'd rather spend the extra $/performance on a platform I don't have as much support time invested. If you love tinkering with it, fine. I'd rather buy it and stay hands off as much as possible.

Share this post


Link to post
12 hours ago, BRiT said:

This isn't about unraid, more of general rant...

 

At this point, I'd rather have an option to enable and disable the patches which are focused and directed on cloud-hosted servers or servers that would be targeted and have actual secure data to protect. Why should normal consumers be forced to run performance heavy security patches? They should be set as a compliance level, like DOD.

 

General security patches are fine since they dont come with 30% perf hits.

 

I must admit I agree, I suspect the actual risk of me running a vulnerable home server is infinitismally small, however that isn't a criticism of LT, as I understand 110% they are kind of "duty-bound" to patch this stuff.

Share this post


Link to post

Oh, absolutely. I don't and can't fault LT for the security patches. LT's prompt responses for security concerns are to be commended. It just seems like the entire software industry is jumping through so many hoops for what are infinitely small possible attack vectors on consumer devices.

Share this post


Link to post
Posted (edited)

https://www.phoronix.com/scan.php?page=article&item=mds-zombieload-mit&num=5

 

"If looking at the geometric mean for the tests run today, the Intel systems all saw about 16% lower performance out-of-the-box now with these default mitigations and obviously even lower if disabling Hyper Threading for maximum security. The two AMD systems tested saw a 3% performance hit with the default mitigations. While there are minor differences between the systems to consider, the mitigation impact is enough to draw the Core i7 8700K much closer to the Ryzen 7 2700X and the Core i9 7980XE to the Threadripper 2990WX."

 

Devastating.

 

I hope everyone starts to buy threadripper/epyc2/ryzen3000.

 

Intel does not deserve people's hard earned money.

 

I guess we need switches to disable on our AMD systems. Due to intel's mistakes. So we aren't caught in the crossfire.

Edited by Dazog

Share this post


Link to post
47 minutes ago, Dazog said:

I hope everyone starts to buy threadripper/epyc2/ryzen3000. 

 

Intel does not deserve people's hard earned money.

At least Intel admits their mistakes, AMD has never chimed in on an obvious power issue with Ryzen:

https://bugzilla.kernel.org/show_bug.cgi?id=196683#c602

 

 

50 minutes ago, Dazog said:

I guess we need switches to disable on our AMD systems. Due to intel's mistakes. So we aren't caught in the crossfire.

The Kernel knows what CPU you are using and adjusts appropriately.

 

Share this post


Link to post
Posted (edited)
14 hours ago, limetech said:

There are several ways to disable the various mitigations, both via kernel parameters and at run-time (though this will require us to include 'debugfs' which we can do).  Google 'linux disable meltdown spectre zombieland mitigations' for a number of how-to articles.  If it gets serious enough we'll probably add debugfs support and perhaps a config page.  Having all these mitigations in place is mainly a C.Y.A. move in my opinion.

I would also love to disable these as well. I got bigger fish to fry than cloud hosted VM vulnerabilities for my little home servers🙂

 

Much more important is my vm with pass through performance, Plex, and handbrake type conversions.

Edited by cybrnook

Share this post


Link to post
Posted (edited)

You can also use this tool to check what vulnerabilities your CPU is susceptible to.

 

 

Edited by ezhik

Share this post


Link to post
Posted (edited)
18 hours ago, limetech said:

There are several ways to disable the various mitigations, both via kernel parameters and at run-time (though this will require us to include 'debugfs' which we can do).  Google 'linux disable meltdown spectre zombieland mitigations' for a number of how-to articles.  If it gets serious enough we'll probably add debugfs support and perhaps a config page.  Having all these mitigations in place is mainly a C.Y.A. move in my opinion.

@limetech

 

Okay, so I did a bit a googling, and to build up, this is what I found based on Kernel level and availability (Leaving zombieload off the table for now since latest release does not yet mitigate it, that will be next):

 

Kernel 4.15
Spectre v2 (CVE-2017-5715) - "nospectre_v2"

 

Kernel 4.17
Spectre v4 (CVE-2018-3639) - "nospec_store_bypass_disable"

 

Kernel 4.19
PR_SPEC_DISABLE_NOEXEC - During compilation
Spectre v1 (CVE-2017-5753) - "nospectre_v1"
Spectre v2 (CVE-2017-5715) - "nospectre_v2"
Spectre v4 (CVE-2018-3639) - "nospec_store_bypass_disable"

 

So checking uname on the latest 6.7.0 (non RC1), I can see we are at 4.19.41, so all three should be available nospectre_v1,v2, and store_bypass_disable. Of course seems there is a flag that can be set during compile, but that's not even worth getting into since that will never happen understandably.

 

Then I found a phoronix forum thread that went back and forth, and I ultimately came out with this:

 

pti=off spectre_v2=off l1tf=off nospec_store_bypass_disable no_stf_barrier

 

 

Where would be the best place to do this in Unraid during boot? Would it simple be an append in the syslinux file?

 

append pti=off spectre_v2=off l1tf=off nospec_store_bypass_disable no_stf_barrier

 

 

EDIT: Seems zombieload (aka MDS) will be mds=off

 

EDIT2: I am testing adding this to my unraid backup servers syslinux.cfg files boot section, like this:

default menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label Unraid OS
  menu default
  kernel /bzimage
  append initrd=/bzroot pti=off spectre_v2=off l1tf=off nospec_store_bypass_disable no_stf_barrier
label Unraid OS GUI Mode
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui
label Unraid OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Unraid OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest

So far I have rebooted with the change, and boot was successful. I am working now to validate these are actually disabled.

 

EDIT3: Okay, it worked. Here is my before and after:

 

BEFORE:

cat /sys/devices/system/cpu/vulnerabilities/*
Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable
Mitigation: PTI
Mitigation: Speculative Store Bypass disabled via prctl and seccomp
Mitigation: __user pointer sanitization
Mitigation: Full generic retpoline, IBPB: conditional, IBRS_FW, STIBP: conditional, RSB filling


AFTER:

cat /sys/devices/system/cpu/vulnerabilities/*
Mitigation: PTE Inversion; VMX: vulnerable
Vulnerable
Vulnerable
Mitigation: __user pointer sanitization
Vulnerable, IBPB: disabled, STIBP: disabled

 

You will notice "conditional cache flushes", "SMT vulnerable", "PTI", "Speculative Store Bypass disabled via prctl and seccomp", "Full generic retpoline"* are all disabled now. 😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁

 

I don't know about you, but I don't care if someone is watching me play games on a vm, or is watching that I am encoding or decrypting a movie, big deal, not much going on at my house anyone would work hard enough to watch....... and if someone did make it that far to target me, I got bigger problems than speculative execution, like checking my firewall rules!!

 

I will post this in another thread for better visibility if you all agree? 

Edited by cybrnook
EDIT2: testing it

Share this post


Link to post

+1
Absolutely vital i would say to be able to disable all these mitigations for non exposed servers or whatever.
Couldnt care less for my server. I just need as much perf as the cpu can deliver. But hats of for LT for beeing on top of this. Respect.

Share this post


Link to post
1 hour ago, cybrnook said:

I will post this in another thread for better visibility if you all agree?  

Nice work!  Right, I think a better place would be in the Security Board.  If you want to add the post there, I'll make it 'sticky'.

 

Also, thanks to all testing this out.  We have no choice but to keep marching on with new kernel releases.

  • Like 1
  • Upvote 1

Share this post


Link to post
Just now, limetech said:

Nice work!  Right, I think a better place would be in the Security Board.  If you want to add the post there, I'll make it 'sticky'.

 

Also, thanks to all testing this out.  We have no choice but to keep marching on with new kernel releases.

Will do, thanks! I will work on working it up a bit nicer and then post once it's super clear.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.