limetech

Administrators
  • Posts

    10184
  • Joined

  • Last visited

  • Days Won

    196

Everything posted by limetech

  1. More clarifications: in Unraid OS only user-defined pools can be configured as multi-device ZFS pools. You can select ZFS as the file system type for an unRAID array disk, but will always be just a single device. The best way to think of this, anywhere you can select btrfs you can also select zfs, including 'zfs - encrypted' which is not using zfs built-in encryption but simply LUKS device encryption. Also note that ZFS hard drive pools will require all devices in a pool to be 'spun up' during use. IMO where ZFS will shine is in large flash-based pools (SSD, NVMe, etc).
  2. 6.12 beta is on 6.0.15 as I type this. OpenZFS is not listed as good to go on 6.1, though looks like that is imminent, at which time we'll upgrade to 6.1.
  3. You really want more than 30 devices in a single array?
  4. Some clarification... Currently: We have a single "unRAID" array(*) and multiple user-defined "cache pools", or simply "pools". Data devices in the unRAID array can be formatted with xfs, btrfs, or reiserfs file system. A pool can consist of a single slot, in which case you can select xfs or btrfs as the file system. Multi-slot pools can only be btrfs. What's unique about btrfs is that you can have a "raid-1" with an odd number of devices. With 6.12 release: You will be able to select zfs as file system type for single unRAID array data disks. Sure, as a single device lots of zfs redundancy features don't exist, but it can be a target for "zfs receive", and it can utilize compression and snapshots. You will be able to select zfs as the file system for a pool. As mentioned earlier you will be able to configure mirrors, raidz's and groups of those. With future release: The "pool" concept will be generalized. Instead of having an "unRAID" array, you can create a pool and designate it as an "unRAID" pool. Hence you could have unRAID pools, btrfs pools, zfs pools. Of course individual devices within an unRAID pool have their own file system type. (BTW we could add ext4 but no one has really asked for that). Shares will have the concept of "primary" storage and "cache" storage. Presumably you would assign an unRAID pool as primary storage for a share, and maybe a btrfs pool for cache storage. The 'mover' would then periodically move files from cache to primary. You could also designate maybe a 12-device zfs pool as primary and 2-device pool as cache, though there are other reasons you might not do that.... * note: we use the term "unRAID" to refer to the specific data organization of an array of devices (like RAID-1, RAID-5, etc). We use "Unraid" to refer to the OS itself.
  5. Our plan is to release a public beta soon(tm) which includes OpenZFS support and changes which Plugin authors need to be aware of. Posting this now as a sneak peak, more detail will follow. That said.... ZFS support: this will let you create a named pool similar to how you can create named btrfs pools today. You will have choice of various zfs topologies depending on how many devices are in the pool. We will support single 2, 3, and 4-way mirrors, as well as groups of such mirrors (a.k.a., raid10). We will also support groups of raidz1/raidz2/raidz3. We will also support expansion of pools by adding additional vdev of same type and width to existing pool. Also will support raid0. It's looking like first release we will support replacing only single devices of a pool at a time even if the redundancy would support replacing 2 or 3 at time - that support will come later. Initially we'll also have a semi-manual way of limiting ARC memory usage. Finally, a future release will permit adding hot spares and special vdev's such as L2ARC, LOG, etc. and draid support. webGUI change: there are several new features but the main change for Plugin authors to note is that we have upgraded to PHP v8.2 and will be turning on all error, warning, and notices. This may result in some plugins not operating correctly and/or spewing a bunch of warning text. More on this later... By "public release" we mean that it will appear on the 'next' branch but with a '-beta' suffix. This means only run on test servers since there may be data integrity issues and config tweaks, though not anticipating any. Once any initial issues have been sorted, we'll release -rc1.
  6. This is nonsense but beyond the scope of the discussion dealing with CRC/DMA errors. For sure, 100%, DMA/CRC errors are hardware faults, not caused by software, file systems, etc. They are reported by physical controllers and indicate a physical h/w problem. In my experience, these kinds of errors commonly originate with bad cables or connectors, or simply faulty components. Another overlooked cause is faulty or overloaded power supplies. Back when we offered server products, we always were careful to source single-rail PSU's so that full capacity of the power supply can be fed to the hard drives. Servers with multi-rail PSU's might have a high overall wattage rating, but any one rail is a fraction of that; and, typically one rail would serve the entire hard drive array. I'm sure you can deduce what the problem is with this arrangement. I haven't looked at many low-level Linux device drivers for several years but I'll take a look at a few and see if they retry CRC/DMA errors. Adding retry logic in md/unraid driver might be something for us to consider. As has been stated correctly, Unraid only disables devices which fail writes because what else can you do if a write fails (and presumably all retries fail)? But sure, if there is a lot of other activity in the server causing a transient dip in voltage, then maybe a retry would succeed.
  7. Respectfully, please refrain from using that term, it's not helpful. We can continue this discussion if you want, I am open to an honest technical exchange, and making code changes if necessary. Some brief comments: re: CRC errors: those typically indicate some kind of corruption in the data path between the device controller and the device itself. Usually as a result of bad cables, connectors, or power supply issues. In general Unraid relies on the Linux device drivers to handle retries and assumes if a write has failed, then the driver and the device itself has exhausted all attempts at recovery and it would be pointless to waste more time re-issuing the same command over and over. re: SMART: it's well known that many drives fail which have perfectly clean SMART reports. The fact you see a disabled drive and the only thing in the SMART report is a single CRC error is suspect.
  8. How did you reboot the server following download/install of new Unraid OS version? Console messages imply an unclean reboot...
  9. I think those are related to ZFS. https://github.com/openzfs/zfs/issues/11691
  10. As always, prior to updating, create a backup of your USB flash device: "Main/Flash/Flash Device Settings" - click "Flash Backup". Bugs: If you discover a bug or other issue in this release, please open a Stable Releases Bug Report. Version 6.11.5 2022-11-20 This release reverts a change which modified the wrong file. This resulted in not being able to select 'macvlan' custom docker network type. New installations only will now have 'ipvlan' selected by default. Docker fix: Set IPVLAN as default only for new installations.
  11. As always, prior to updating, create a backup of your USB flash device: "Main/Flash/Flash Device Settings" - click "Flash Backup". Bugs: If you discover a bug or other issue in this release, please open a Stable Releases Bug Report. Version 6.11.4 2022-11-18 This release is focused on bug fixes and minor improvements. Diagnostics Fix share information in Diagnostics. Docker docker: version 20.10.21 (CVE-2022-39253 CVE-2022-2879 CVE-2022-2880 CVE-2022-41715) Fix: "please wait" message after removing orphan image. Set IPVLAN as default for new installations. Samba samba: version 4.17.3 (CVE-2022-42898) Set the default "max open files" to the value of 'ulimit -n' which is currently set to 40960. VM Manager Allow Network PCI devices to have boot order. Change to use libvirt_domain_interface_addresses which has been available for a long time but was not documented in the API manual. Using this function stops tainted warnings being written to the log, ex: "Domain id=4 is tainted: custom-ga-command". Fix for VM Rename. Misc Fixed issue which prevented array Autostart with a Trial key. Fixed encoding issue when passwords contain multi-byte UTF-8 characters. Parity History: add day of week to date. Shares: Fix: do not allow creation of hidden share names. Main page: reinitialize disk transfer statistics upon opening new browser session. Management Access page: improved messaging for SSL Firefox: version 107.0.r20221110173214 (AppImage) When all browser sessions close, stop unnecessary background processes. Base Distro bash: version 5.2.009 btrfs-progs: version 6.0.1 gdk-pixbuf2: version 2.42.10 glibc-zoneinfo: version 2022f intel-microcode: version 20221108 libX11: version 1.8.2 libnftnl: version 1.2.4 nghttp2: version 1.51.0 php: version 7.4.33 (CVE-2022-31630 CVE-2022-37454) sed: version 4.9 sysstat: version 12.7.1 xkbevd: version 1.1.5 xkill: version 1.0.6 xlsatoms: version 1.1.4 xlsclients: version 1.1.5 xz: version 5.2.8
  12. Sure you can set a larger value, doesn't mean it uses it You can turn on Samba debugging and see if that "file_init_global: Information only: requested ..." message appears.
  13. Got this from looking at the source: https://github.com/samba-team/samba/blob/master/source3/smbd/files.c#L1401
  14. I normally don't post the gory details of a bug and bug fix/workaround. For the issue where 6.11.2 could not format devices larger than 2TB here is what was the cause and the fix that went into 6.11.3. First, the reason this got published without this issue being caught is that my test server with devices larger then 2TB were already formatted. Other test servers, which are actually VM's had smaller virtual devices and formatting worked ok with them. So sorry about that, sometimes sh*t happens, but this particular bug would not have caused any data loss. Anyway.... The bug was in the 'sgdisk' utility included in the "gptfdisk" package, where version 1.0.9 was released on April 15, 2022. This was integrated into Unraid OS very early in 6.11 development and appeared in 6.11.0 on Sep 23, 2022, where it works fine. But then the libpopt library was updated on Oct 19, 2022, and included in a larger slackware package called "aaa_libraries" which was integrated into Unraid 6.11.2. The libpopt library includes functions used by nearly all C-based utilities to parse command line options. The sgdisk command also uses this library to parse command line options. The bug is that sgdisk was referencing freed memory due to how a function in libpopt was implemented. But this didn't cause problems because that freed memory wasn't reused until after sgdisk already referenced it. Well along comes an upgrade to the libpopt library which frees memory differently which caused sgdisk now to reference garbage data and fail as described. Author of gptfdisk added a fix but for some reason has not published a new release. Edit: the fix was to revert gptfdisk to 1.0.8 which used the earlier version of libpopt, and to also revert libpopt to the earlier version.
  15. As always, prior to updating, create a backup of your USB flash device: "Main/Flash/Flash Device Settings" - click "Flash Backup". Bugs: If you discover a bug or other issue in this release, please open a Stable Releases Bug Report. Version 6.11.3 2022-11-08 This release is focused on bug fixes and minor improvements. In particular, we need to revert a base library due to a bug which prevents formatting devices >2TB in size. Management Reverted 'libpopt.so.0.0.1' to workaround 'sgdisk' bug used to format devices larger than 2TB. Fixed issue where description setting for 'root' user was not preserved across reboots. Fix issue that sometimes information window opens empty. VM Manager Set correct values when edit for USB Boot and correct Share Selection. Add Nic boot order. Base Distro gptfdisk: version 1.0.8 (revert from version 1.0.9) libXext: version 1.3.5 libXinerama: version 1.1.5 libdrm: version 2.4.114 libedit: version 20221030_3.1 mcelog: version 190 ntfs-3g: version 2022.10.3 openssl: version 1.1.1s openssl-solibs: version 1.1.1s pixman: version 0.42.2 sessreg: version 1.1.3 sudo: version 1.9.12p1 xsetroot: version 1.1.3
  16. Thank you for your post. It's unfortunate that some feel necessary to "bad mouth" other products. We've been around a long time and during all that time I have never denigrated any other NAS or NAS-like products, and AFAIK no one associated with Lime Tech has either. My view of things is that it's a big market out there and running a business (and staying in business) is very difficult, I'll give kudos to anyone attempting it. Not only that, competition is good, it keeps everyone busy innovating. re: TruNas - this is produced by iXsystems which makes its money in the Enterprise space. Pretty sure they would not endorse comments such as those. Again, thank you and for sure you hit on the true value of Unraid OS: this Community! Actually sales have been increasing
  17. The 6.11.2 release is focused on bug fixes and minor improvements. Please note there are important security mitigations in Samba, all users are encouraged to update. As always, prior to updating, create a backup of your USB flash device: "Main/Flash/Flash Device Settings" - click "Flash Backup". Bugs: If you discover a bug or other issue in this release, please open a Stable Releases Bug Report. Version 6.11.2 2022-11-04 This release is focused on bug fixes and minor improvements. Please note there are important security mitigations in Samba, all uses are encouraged to update. VM Manager If you boot a VM from a passed physical USB device there is a new setting called Enable USB boot which must be set to Yes. This is due to feature change in the latest OVMF component. Changes: Add boot order for USB and USB Boot option. Using 'boot order' field VM can boot from a passed-through NVMe controller/device. Add virtiofs/9p Unraid share mapping. CD Hotplug bug fix. Fix Hotplug within Templates. Fix changing from passthru to custom CPU. Test for guest agent to suppress error message in libvirt log, and show text for guest not running or guest agent is not installed. Fix unable to change HyperV settings. Add 'start with console' option. OVMF for QEMU: version edk2-stable202208 Wireguard Changes: Fixed problem setting up tunnels if using default network.cfg settings. Fix typo setting up routes. Plugin manager Changes: If plugin files cannot be read from CDN URL, fall back to non-CDN URL. Enforce plugin files must end in '.plg'. Fix post hook failing when plugin download fails. Adjust the wget parameters to keep it from indefinitely hanging. Scheduler For scheduled Parity Checks, change default mode to Non-correcting. In case there are disk issues during a parity check it would be safer to have the default be non-correct. If sync errors are detected then the user can analyze the log and act accordingly. Misc. webGUI Changes: DashStats: The inbound networkstats is out of place when numeric value is xxx.x. Docker: Add/clarify Help to Docker Custom Network settings. Docker: Fix display aberration on orphan images SMART attributes: clarify message: "Can not read attributes" to "Attributes not available" ShareEdit/ShareList: show pool not defined when share pool is not a defined pool Linux kernel version 5.19.17 patch: additional NVMe BOGUS_NID quirks for non-compliant devices reported by users sound support: CONFIG_SOUND: Sound card support CONFIG_SND: Advanced Linux Sound Architecture CONFIG_SND_OSSEMUL: Enable OSS Emulation CONFIG_SND_PCM_OSS: OSS PCM (digital audio) API CONFIG_SND_HRTIMER: HR-timer backend support CONFIG_SND_SUPPORT_OLD_API: Support old ALSA API CONFIG_SND_PROC_FS: Sound Proc FS Support CONFIG_IPV6: The IPv6 protocol (change from module to built-in) Base Distro Package updates including CVE mitigations: bash: version 5.2.002 bind: version 9.18.8 (CVE-2022-38178 CVE-2022-38178 CVE-2022-3080 CVE-2022-2795) btrfs-progs: version 6.0 curl: version 7.86.0 dbus: version 1.14.4 freeglut: version 3.4.0 git: version 2.38.1 (CVE-2022-39253 CVE-2022-39260) glibc-zoneinfo: version 2022e harfbuzz: version 5.3.1 icu4c: version 72.1 iproute2: version 6.0.0 kernel-firmware: version 20221017_48407ff less: version 608 libXmu: version 1.1.4 libXrender: version 0.9.11 libedit: version 20221009_3.1 libffi: version 3.4.4 libgpg-error: version 1.46 libpciaccess: version 0.17 libunistring: version 1.1 libxkbfile: version 1.1.1 libxshmfence: version 1.3.1 lsof: version 4.96.4 nginx: version 1.22.1 openssh: version 9.1p1 php: version 7.4.32 pixman: version 0.42.0 rsync: version 3.2.7 samba: version 4.17.2 (CVE-2021-20251 CVE-2022-3437 CVE-2022-3592) sudo: version 1.9.12 tree: version 2.0.4 xkeyboard-config: version 2.37 xterm: version 375 zlib: version 1.2.13 (CVE-2022-37434)
  18. Dear Unraid Community, As we look ahead to 2023 and beyond, all of us at Lime Tech are committed and focused on bringing you the best possible value out of your hardware and software investments. As we continue to expand Unraid OS and our team, we want to ensure that we're prioritizing projects and features that are in line with our customer's needs and desires. We greatly value your feedback and would appreciate you spending 5-10 minutes taking this anonymous survey. As a bonus, you can enter to win one of ten $50 VISA gift cards as a token of our appreciation. Thank you in advance for taking part in shaping the future of Unraid OS! Cheers, Tom & the entire Lime Technology Team
  19. Did you try with ubuntu? If so, please post output of testparm -sv from the ubuntu install that works.
  20. Thank you for your report. I have TM backups running ok with Monterey (12.6). At first I thought it was because the share is marked 'public'. Changed to 'private' and there was some connectivity issues but eventually poked around the TM preferences and got it to work again... so the investigation continues...
  21. How long did you let it sit? bzroot is quite a bit larger in 6.11 vs 6.9