• Unraid OS version 6.10.0-rc4 available


    limetech

    This is primarily a bug fix release.  We have not addressed every issue that has been reported.  In the past we would typically delay releases until most things were addressed; however, we are committed to producing releases as quickly as possible as issues are fixed and small improvements are made.

     

    It would be extremely helpful to us to report issues by creating separate Reports here rather than creating a reply in this topic.

     

    The main issues addressed here have to do with nchan errors, DNS Rebinding Protection check, and XFS formatting issue.

     

     

    6.10.0 Summary of Changes and New Features

     

    As always, prior to updating, create a backup of your USB flash device:  "Main/Flash/Flash Device Settings" - click "Flash Backup".

     

    [rc3] Plugin Authors:  We patched the upgradepkg script to prevent it from replacing an installed package with an earlier version of the same package, i.e., no downgrading.  If a plugin really needs to replace a package with a downgraded version it can include the '--reinstall' option.

     

    Also be sure to check out the Dynamix File Manager plugin available now through Community Apps!

     

    UPC and My Servers Plugin

    The most visible new feature is located in the upper right corner of the webGUI header.  We call this the User Profile Component, or UPC.  The UPC allows a user to associate their server(s) and license key(s) with their Unraid Community forum account, also known as an Unraid.net account.

     

    Starting with this release, it will be necessary for a new user to either sign-in with existing forum credentials or sign-up, creating a new account via the UPC in order to download a Trial key.  All key purchases and upgrades are also handled exclusively via the UPC.

     

    Signing-in provides these benefits:

    1. My Servers Dashboard - when logged into the forum a new My Servers menu item appears. Clicking this brings up a Dashboard which displays a set of tiles representing servers associated with this account.  Each tile includes a link to bring up the servers webGUI on your LAN.  Install the My Servers plugin to provide real-time status and other advanced features (see below).
    2. Notification of critical security-related updates.  In the event a serious security vulnerability has been discovered and patched, we will send out a notification to all email addresses associated with registered servers.
    3. Posting privilege in a new set of My Servers forum boards.
    4. No more reliance on email and having to copy/paste key file URLs in order to install a license key - keys are delivered and installed automatically to your server.

     

    Once a license key has been provisioned, it is not necessary to remain signed-in, though there is no particular reason to sign-out.

     

    My Servers Plugin

    My Servers is what we call our set of cloud-based or cloud-enabled services and features that integrate with your Unraid server(s).  Once installed here are some of the features of My Servers:

    • Real-time Status - with the plugin installed each server tile on the My Servers Dashboard will display real-time status such as whether the server is online or offline, storage utilization and other information. 
    • Remote Access link - if enabled, a link is displayed on the My Servers Dashboard to bring up a server webGUI remotely and over the Internet.
    • Automatic Flash Backup - every registered server is provided with a private git repo initially populated with the contents of your USB flash boot device (except for certain files which contain private information such as passwords).  Thereafter, configuration changes are automatically committed.  A link is provided to download a custom zip file that can be fed as input to the USB Flash Creator tool to move your configuration to a new USB flash device.

     

    My Servers is an optional add-on, installed through Community Apps or via direct plugin URL.  Detailed instructions can be found here.

     

    If you have installed the My Servers plugin, signed-in servers will maintain a websocket connection to a cloud-based Lime Technology proxy server for the purpose of relaying real-time status.

     

    Security Changes

    • It is now mandatory to define a root password.  We also created a division in the Users page to distinguish root from other user names.  The root UserEdit page includes a text box for pasting SSH authorized keys.
    • For new configurations, the flash share default export setting is No.
    • For all new user shares, the default export setting is No.
    • For new configurations, SMBv1 is disabled by default.
    • For new configurations, telnet, ssh, and ftp are disabled by default.
    • We removed certain strings from Diagnostics such as passwords found in the 'go' file.

     

    Virtualization

    Both libvirt and qemu have been updated.  In addition, qemu has been compiled with OpenGL support, and [rc2] ARM emulation (experimental).

     

    [rc2] To support Windows 11 which requires TPM and Secure boot, we have added TPM emulation; and, added a "Windows 11" VM template which automatically selects TPM-aware OVMF bios.  Also, here are instructions for upgrading a Windows 10 VM to Windows 11.  Special thanks to @ich777 who researched and determined what changes and components were necessary to provide this functionality.

     

    The built-in Firefox browser available in GUI-mode boot is built as an AppImage and located in the bzfirmware compressed file system image.  This saves approximately 60MB of RAM.

     

    The Wireguard plugin has been integrated into webGUI, that is, no need for the plugin.  If you had the plugin installed previously, it will be uninstalled and moved to the "Plugins/Plugin File Install Errors" page. No action is needed unless you want to press the Delete button to remove it from that page. Your WireGuard tunnels and settings will be preserved.

     

    Simplified installation of the Community Apps plugin.  The webGUI automatically includes the Apps menu item, and if CA is not already installed, the page offers an Install button.  No need to hunt for the plugin link.

     

    [rc3] Moving to Let's Encrypt wildcard SSL certificates. 

    Starting with this release, we no longer issue new single-host SSL certificates (which we're calling legacy certificates).  Instead, all new Unraid.net SSL certificates are wildcard certificates (still provided by Let's Encrypt).  In order to provision a new wildcard certificate, or upgrade a legacy certificate, you must be signed-in to Unraid.net.  You do not need to be signed-in however, to have either type of certificate automatically renewed when it is within 30 days of expiration.

     

    The URL used to access your server making use of a wildcard certificate has this form:

    • https://[lan-ip].[hash].myunraid.net

    where,

    • [lan-ip] is your severs LAN IP address with dots changed to dashes
    • [hash] is a 40-character hex string (160 bits) unique to this server (and different from similar [hash] in legacy certificates)

    example:

     

    We added a new DDNS server which listens at "myunraid.net".  This server extracts [lan-ip] from the domain name and returns the IP address where the dashes are changed back into dots.  There are several benefits to this approach for both our users and for us:

    • Eliminates DNS propagation delays when you first provision a certificate or when a server LAN IP address (or WAN IP address) changes.  Since the domain name includes the IP address, any IP address change also changes the domain name, hence will not be contained in any intermediate DNS cache.  We also changed the TTL from 1 hour to 7 days further reducing overhead and alleviating issues where someone's internet goes down for brief periods.
    • Improves privacy because your remote access WAN IP address can't be determined by simply prepending "www" to your local access URL.
    • Moves DNS functionality off the 'unraid.net' domain and isolates it on 'myunraid.net' domain.

     

    In previous releases code that provisions (allocates and downloads) an Unraid.net SSL certificate would first test if DNS Rebinding Protection was enforced on the user's LAN; and, if so, would not provision the certificate.  Since there are other uses for a LE certificate we changed the code so that provision would always proceed.  Next, we changed the logic behind the Auto selection of "Use SSL/TLS" setting on the Management Access page.  Now it is only possible to select Auto if both a LE certificate has been provisioned and DNS Rebinding Protection is not enforced.  This is a subtle change but permits certain My Servers features such as Remote Access.

     

    Upon upgrading, you will need to modify any server bookmarks with new the URL; however, if you server is signed-in to Unraid.net then the My Servers dashboard maintains the correct Local Access URL for each of your servers.

     

    More information including use cases may be found in Documentation here.

     

    Linux Kernel

    Upgrade to [rc4] Linux 5.15.30 kernel which includes so-called Sequoia and Dirty Pipe vulnerability mitigations.

     

    In-tree GPU drivers are now loaded by default if corresponding hardware is detected:

    • amdgpu
    • ast
    • i915
    • radeon

     

    These drivers are required mostly for motherboard on-board graphics used in GUI boot mode.  Loading of a driver can be prohibited by creating the appropriate file named after the driver:

    echo "blacklist i915" > /boot/config/modprobe.d/i915.conf

    Alternately, the device can be isolated from Linux entirely via the System Devices page.  Note that in Unraid OS 6.9 releases the in-tree GPU drivers are blacklisted by default and to enabling loading a driver you need to create an empty "conf" file.  After upgrading to Unraid OS 6.10 you may delete those files, or leave them as-is.  This change was made to greatly improve the Desktop GUI experience for new users.

     

    Added support for Intel GVT-g, which lets you split your Intel i915 iGPU into multiple virtual GPUs and pass them through to multiple VMs, using @ich777's Intel-GVT-g plugin.

     

    Added support for gnif/vendor-reset.  This simplifies @ich777's AMD Vendor Reset plugin which permits users to get their AMD video cards to reset properly.

     

    [rc2] Added so-called "add-relaxable-rmrr-5_8_and_up.patch" modified for our kernel
      https://github.com/kiler129/relax-intel-rmrr/blob/master/patches/add-relaxable-rmrr-5_8_and_up.patch

    Thanks to @ich777 for pointing this out.

     

    [rc2] Enabled additional ACPI kernel options
    [rc2] Enabled TPM kernel modules (not utilized yet) - note this is for Unraid host utilizing physical TPM, not emulated TPM support for virtual machnes.

    [rc4] Updated out-of-tree drivers

     

    Base Packages

    Virtually the entire base package set has been updated.

     

    [rc2] For SMB: Samba version 4.15 SMB3 multi-channel is no longer marked "experimental" and is enabled by default.

     

    [rc2] Per request we added the mcelog package.  With inclusion of this package, if you have an AMD processor you may see this error message in the system log:

    mcelog: ERROR: AMD Processor family 23: mcelog does not support this processor. Please use the edac_mce_amd module instead.

    We're not sure what to make of this.  It appears mcelog is being deprecated in favor of rasdaemon.  This is something we need to research further.

     

    Other improvements available in 6.10, which are maybe not so obvious to spot from the release notes and some of these improvements are internal and not really visible:

     

    Event driven model to obtain server information and update the webGUI in real-time

    • The advantage of this model is its scalability. Multiple browsers can be opened simultaneously to the webGUI without much impact
    • In addition stale browser sessions won't create any CSRF errors anymore
    • People who keep their browser open 24/7 will find the webGUI stays responsive at all times
    • [rc3] Consistent state information is maintained across all browser instances open to a particular server

     

    Docker labels

    • Docker labels are added to allow people using Docker compose to make use of icons and GUI access
    • Look at a Docker 'run' command output to see exactly what labels are used

     

    Docker custom networks

    • A new setting for custom networks is available. Originally custom networks are created using the macvlan mode, and this mode is kept when upgrading to version 6.10
    • The new ipvlan mode is introduced to battle the crashes some people experience when using macvlan mode. If that is your case, change to ipvlan mode and test. Changing of mode does not require to reconfigure anything on Docker level, internally everything is being taken care off.

     

    Docker bridge network (docker0)

    • docker0 now supports IPv6. This is implemented by assigning docker0 a private IPv6 subnet (fd17::/64), similar to what is done for IPv4 and use network translation to communicate with the outside world
    • Containers connected to the bridge network now have both IPv4 and IPv6 connectivity (of course the system must have IPv6 configured in the network configuration)
    • In addition several enhancements are made in the IPv6 implementation to better deal with the use (or no-use) of IPv6

     

    Plugins page

    • The plugins page now loads information in two steps. First the list of plugins is created and next the more time consuming plugin status field is retrieved in the background. The result is a faster loading plugins page, especially when you have a lot of plugins installed

     

    Dashboard graphs

    • The dashboard has now two graphs available. The CPU graph is displayed by default, while the NETWORK graph is a new option under Interface (see the 'General Info' selection)
    • The CPU graph may be hidden as well in case it is not desired
    • Both graphs have a configurable time-line, which is by default 30 seconds and can be changed independently for each graph to see a longer or shorter history.
    • Graphs are updated in real-time and are useful to observe the behavior of the server under different circumstances

     

    Scheduler Improvements

    • [rc3] You can now split a parity check into smaller pieces and let it run over multiple days or weeks.  For example a check can be performed in a time frame of 01:00am to 06:00am for several days in a row until it is completed.  This way a long parity check won’t interfere with the normal day activities, like watching a movie.
    • [rc3] Added ability to schedule pool 'balance' and 'scrub' operations and calculate whether a full balance is recommended.

     

    Other Changes

    • We switched to a better-maintained version of the WSD server component called wsdd2 in an effort to eliminate instances where the wsd daemon would start consuming 100% of a CPU core.  [rc2] Automatically restrict wsdd to listen only at the primary network interface (br0, bond0, or eth0, depending on config).
    • Fixed issue where you couldn't create a docker image on a share name that contains a space.
    • Fixed issue where 'mover' would not move to a pool name that contains a space.
    • Fixed issue in User Share file system where permissions were not being honored.
    • We increased the font size in Terminal and [rc2] fixed issue with macOS Monterey.
    • [rc2] Fixed jumbo frames not working.
    • [rc2] sysctl: handle net.netfilter.nf_conntrack_count max exceeded (increase setting to 131072) - hattip to Community Member @DieFalse

    • [rc2] Mover will create '.partial' file and then rename upon completion.

    • [rc2] Check bz file sha256sums at boot time.

    • [rc3] Fixed bug found by @thohell where md_sync_limit was not being honored to limit stripe_head cache usage when other I/O is active.  The effect of this fix is to drastically slow down parity operations if other I/O is happening (such as streaming a video).  Throttling of parity sync operations can be adjusted by changing the 'Settings/Disk Settings/Tunable (md_sync_limit)' value.

    • [rc3] Fixed btrfs pool device replace corner cases.  Important note: if you 'unassign' a device from a btrfs multiple-device pool, and that device is still physically present, upon array Start we will erase the LUKS header on the device if present, and delete the partition structure, thereby effectively erasing all the data contained on the device.  This is necessary in order to convince btrfs to no longer use the device and to free it for assignment to another pool.

    • [rc3] For cookies managed by webGUI, changed sameSite cookie attribute from 'strict' to 'lax'.  This change was made to solve an issue with Terminal window not opening in Safari.

     

    Credits

    Special thanks to all our beta testers and especially:

    @bonienl for his continued refinement and updating of the Dynamix webGUI.

    @Squid for continued refinement of Community Apps and associated feed.

    @dlandon for continued refinement of Unassigned Devices plugin and patience as we change things under the hood.

    @ich777 for assistance and passing on knowledge of Linux kernel config changes to support third party drivers and other kernel-related functionality via plugins.

    @SimonF for refinements to System Devices page and other webGUI improvements.  We intend to merge your mover progress changes during this RC series.

    @thohell for an extra set of eyes looking at md/unraid driver and for work-in-progress of adding changes to support multiple Unraid arrays.

     


     

    Version 6.10.0-rc4 2022-03-19 (vs. 6.10.0-rc3)

    Base distro:

    • docker: version 20.10.13
    • firefox: 98.0.r20220313140707 (AppImage)
    • kbd: version 1.15.3 (to support non-US keyboards)

    Linux kernel:

    • Linux 5.15.30-Unraid
    • oot: md/unraid: version 2.9.22 (revert setting min sector size to 4096)
    • oot: added Intel ixgbe: version 5.14.6

    Management:

    • emhttpd: add 'rootshare' reserved name
    • rc.nginx: prefer IPv4 if both IPv4 and IPv6
    • rc.nginx: ignore case in processing Subject field for custom certificates
    • rc.nginx: remove default server block returning 404 for https if USE_SSL==no and no CA-signed cert
    • webgui: Docker: Add Network / Privacy Category
    • webgui: Revert back to default capitalization of device names
    • webgui: Fix PHP error when calculating balance level
    • webgui: Docker: make popup window fit in browser window
    • webgui: Change parity sync notification from error to notice level
    • webgui: Changed header selection for better support of Android
    • webgui: Let setting "showBannerGradient" default to "yes"
    • webgui: Remove Nchan error detection (Rely on the automatic reconnect of Nchan to re-establish connections when communication is slow)
    • webgui: Fix: Improved DNS Rebinding checks
    • webgui: Revised filedrop.js
    • webgui: Use https for internet connectivity check
    • webgui: Fix regression error for themes auzre & gray
    • webgui: Highlight selected row when hovering over array or shares
    • Like 9
    • Thanks 4



    User Feedback

    Recommended Comments



    3 hours ago, blaine07 said:

    When sever came back online it kicked off a parity check on its own. Is that normal behavior for RC4? I don’t think I’ve ever seen it initiate parity check on a clean reboot before?

    That means Unraid thinks the reboot was not actually clean :( 

     

    You might find this section of the online documentation that can be accessed via the Manual link at the bottom of the GUI to be of use?

     

     

    Link to comment

    I am doing a file transfer which is going pretty slowy, only about 600GB.  If I stop the array then reboot will File Manager resume after the upgrade or is it best to wait until the transfer is done before rebooting?

    Edited by mrkenn1
    Link to comment
    29 minutes ago, mrkenn1 said:

    I am doing a file transfer which is going pretty slowy, only about 600GB.  If I stop the array then reboot will File Manager resume after the upgrade or is it best to wait until the transfer is done before rebooting?

     

    The File Manager does not resume an operation, after rebooting the system it will start a new operation. In other words it starts again copying or moving the interrupted file.

     

    Link to comment

    I noticed that when I upgraded from 6.9.2 to 6.10 - RC4 many of my docker containers were throwing various errors. I had to remove and reinstall a few of them to correct the problem. I had issues with Cloudberry Backup, Sonarr (linuxserver), Radarr (linuxserver), Krusader (ich777) and Plex (linuxserver). In addition for plex I had to correct the permissions on the Appdata folder as it could not read the database.

    Edited by mackid1993
    grammar
    Link to comment
    50 minutes ago, mackid1993 said:

    I noticed that when I upgraded from 6.9.2 to 6.10 - RC4 many of my docker containers were throwing various errors. I had to remove and reinstall a few of them to correct the problem. I had issues with Cloudberry Backup, Sonarr (linuxserver), Radarr (linuxserver), Krusader (ich777) and Plex (linuxserver). In addition for plex I had to correct the permissions on the Appdata folder as it could not read the database.

    Same here - not sure if it happened in rc3 or rc4 though.

    Changing ownership of the appdata files (just for misbehaving dockers) to nobody:users seemed to fix it for me.

     

    Link to comment

    Just upgraded from 6.9.2 to 6.10 - RC4 and it seems that all my ACLs are broken.

     

    It seems like they are stored and shown but not actually used. Example ACL from one of my directories:

     

    root@nas:/mnt/user# getfacl users
    # file: users
    # owner: administrator
    # group: domain\040admins
    user::rwx
    user:domain\040admins:rwx
    user:domain\040users:r-x
    group::rwx
    group:administrator:rwx
    group:domain\040admins:rwx
    group:domain\040users:r-x
    mask::rwx
    other::---

     

    None of my users except the administrator can access this directory even though they are in the "Domain Users" group. Domain join and user lookup still seems to be working and user is definitley in the according group:

     

    root@nas:/mnt/user# id g0dscookie
    uid=1461716048(g0dscookie) gid=1461715457(domain users) groups=1461715457(domain users),[..snip..]

     

    Also I cannot access my Unraid Server via IP nor local domain (i.e. https://nas.example.org:8443) anymore since the nginx config now has a catch-all with return 404 in place. Only [hash].unraid.net has a server block which has access to the Web GUI.

     

    After downgrading ACLs are working again.

    Link to comment
    3 minutes ago, annihilatethee said:

    Upgraded from rc2 to rc4 and like a previous post mine was also considered an unclean reboot.

    Yeah I think mine may have been my flash drive dying. Had to change it yesterday. I think all the writing for update just happened to kill my drive lol. Just my kind of luck though LOL 

    Link to comment
    1 minute ago, blaine07 said:

    Yeah I think mine may have been my flash drive dying. Had to change it yesterday. I think all the writing for update just happened to kill my drive lol. Just my kind of luck though LOL 

    You are not alone 😭

    This morning RC4 also killed my flash drive (beyond repair!). I had quite a hard time to get the box up again (of course it now tells me "come back tomorrow when the parity check is done").

    The whole flash thing is the most stupid idea of UNRAID. I do understand its purpose, but adding expensive parity drives, mirroring expensive cache drives and then you depend helplessly on a 3€ stick that was never built with the idea of running 24/7 for years...

    (yeah, I know, its for the licence, but this is still very stupid).

    I'm back to 6.9.2 now and have no further plans on updating soon (the only thing I "see" that is missing now is the CPU temperature reading. 6.9.2 is not able to handle a Ryzen 5000).

     

    • Haha 1
    Link to comment
    5 minutes ago, Michael Meiszl said:

    you depend helplessly on a 3€ stick that was never built with the idea of running 24/7 for years...

    This isn’t true. UnRAID runs from ram. The only time the flash is accessed is when settings/base system are changed.

    Link to comment
    1 minute ago, wgstarks said:

    This isn’t true. UnRAID runs from ram. The only time the flash is accessed is when settings/base system are changed.

    yeah I know, but this does not matter. The stick is used if you change config (like installing a plugin like I did this morning) and even if you dont touch it at all, keeping it under current for a long period will kill most of those beasts already.

    And you only notice it one day if you dare to reboot...

    So this is a ticking bomb in your server.

     

    Link to comment
    1 hour ago, Michael Meiszl said:

    So this is a ticking bomb in your server.

     

    I am still using my original USB stick of 12 years ago, it has actually survived several hard disks in terms of longevity ...

     

    • Thanks 1
    Link to comment
    6 minutes ago, bonienl said:

     

    I am still using my original USB stick of 12 years ago, it has actually survived several hard disks in terms of longevity ...

     

    going on 7 years for me

    • Thanks 1
    Link to comment

    Since the upgrade I have 2 git processes running @ 100% pretty much full time.  I've not seen this prior to rc4, but the server has been up for 2 days and showing this.  sometimes these stop for a minute or so before restarting.  

    Link to comment

    Booting from USB is pretty common in this type of system ... ESXI is another example.

     

    As for reliability, the trick might be not to rely on a 3 dollar stick... I've run for 4 years on one and swapped it out to a new one following Spaceinvader Ones findings:

     

     

    Arbadacarba

    Link to comment

    Sorry guys, my intent wasn’t to de-rail thread about USB. I just happen to have poor luck. 😞

    Link to comment
    6 hours ago, Michael Meiszl said:

    helplessly on a 3€ stick

    Not trying to discount your experience but I haven't had to change my USB stick for more than 5 years since I moved to one of the recommended ones.

    Link to comment

    My flash drive has gotten corrupted a few times (not sure why; happened when upgrading to rc3 too), but its still the same drive itself. A chkdsk on a Windows machine gets me back up and running pretty quick.

    Link to comment

    yeah, but there is still the old Murphy hanging around.. SHIT HAPPENS.

    My stick was really broken, not fixable anymore.

    The problem today is that its already hard to order a small one and then you dont know, if UNRAID will accept it or not. So I ended up ordering 10 different sticks yesterday in the hope that one of them has a usable UUID.

     

    Got a bit more expenisve now, 3,29€ this time 🙂

     

    But the point still is: the stick is a SPoF (single point of failiure) which cannot be overcome unless you buy a 2nd licence and prepare a 2nd stick with it putting it into the vault until the 1st one breaks...

     

    Link to comment
    10 minutes ago, Michael Meiszl said:

    So I ended up ordering 10 different sticks yesterday in the hope that one of them has a usable UUID.

    I would recommend that you use a Transcend 32GB USB 2.0, you can get them over here in Europe pretty easily and they have a valid UUID.

     

    Never had a single issue with this devices.

    Link to comment
    7 minutes ago, ich777 said:

    recommend that you use a Transcend 32GB USB 2.

    ok, I've just ordered 5 of them... but I thought 32Gb were too big for UNRAID?

     

    Link to comment
    12 minutes ago, Michael Meiszl said:

    ok, I've just ordered 5 of them... but I thought 32Gb were too big for UNRAID?

     

    Definitely fine to use as my 32GB stick shows 30.1 GB free.

    Link to comment
    22 minutes ago, Michael Meiszl said:

    ok, I've just ordered 5 of them... but I thought 32Gb were too big for UNRAID?

    Look at this site and scroll down to the Hardware Requirements: Click

    Link to comment



    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.