TheLinuxGuy

+1 - I am shocked this FR from 2014 still untouched. Now that there is ZFS available on pools and cache... why can't we use the "ZFS dataset" "quota" settings for this?

Would you then consider then this feature request to be valid? I appreciate that you helped clarify one of the existing unraid features that I wasn't aware of; I don't think the previous posts troubleshooting addressed my Feature Request for more flexibility to control how the cache actually works in each share. My other feature request is also related to the cache and using quotas rather than waiting for the entire cache disk drive to fill up.

If I set this to 4Gb (matching my nvme cache mergerfs setting), is it possible to ensure each array disk (non-cache) to have a different min space reservation (300GB)?

I think the way this is setup is a bit confusing; the help text for that setting is not cache specific (so it feels ambiguous) Why not have a separate file setting for this "Primary" and "Secondary" storages? This is how mergerfs is flexible in. In mergerfs - I configure my slow disks ("array" in unraid parlace) with that setting; while my "cache" has a different configuration. Note how I ensure each individual array disk must always keep 300G free before the next candidate is chosen for writes. NVME cache has 4G reservation. Could we get this flexibility in unraid? Cache and Secondary storage settings per disk member of each. # add fstab slow-storage /mnt/disk* /mnt/slow-storage fuse.mergerfs defaults,nonempty,allow_other,use_ino,category.create=msplfs,cache.files=off,moveonenospc=true,dropcacheonclose=true,minfreespace=300G,fsname=mergerfs 0 0 # add fstab /cache nvme /cache:/mnt/slow-storage /mnt/cached fuse.mergerfs defaults,nonempty,allow_other,use_ino,noforget,inodecalc=path-hash,security_capability=false,cache.files=partial,category.create=lfs,moveonenospc=true,dropcacheonclose=true,minfreespace=4G,fsname=mergerfs 0 0

While copying data to my new unraid server to a share which has cache+array I keep my writes denied once the cache fills up. This is an inconvenience for me - if there's space in the secondary storage the writes should instead go to the array until mover frees up the cache. The server I am moving from is my mergerfs + snapraid setup (https://github.com/TheLinuxGuy/free-unraid) and this is absolutely one of the best features of mergerfs (https://github.com/trapexit/mergerfs). I am not sure why unraid doesn't have it on its FUSE implementation so here's a feature request for it.

Posting this question in 2023 because some of the resources I found are a few years old and it seems that unraid 6.12 may have improved how 'macvlan' and docker works? My goal: I would like certain (most) of my containers running on unraid to get a DHCP hostname and dedicated IP from my router. I think the macvlan docker docs are a bit confusing; there is a setting for allocating an IP range from my network but I don't think that does what I want which is to have the container get its IP from my DHCP pool (router - so it also resolves container-name.local)

Unraid should give its users the ability to "split" a cache disk with specific settings or limits per unraid-share. For example, a ZFS cache disk named "cache" of 2TB raw disk-size. Two shares may exist with cache as primary storage, secondary storage is the array for them. One of these shares is a heavy use media share. The user only wants "media" share to consume a maximum of 400GB out of the total 2TB disk. I was thinking this could easily be achieved with ZFS quotas and a dataset that's a children of the 2TB disk.... but I do not think unraid FUSE knows about zfs quotas etc... so that's my feature request... allow your users to fragment a cache with limits that are not based off the raw total storage of a cache disk but user-default prior to sending writes to secondary storage.

+1 to a smart "auto-tiered storage" model. btrfs FR: https://github.com/kdave/btrfs-progs/issues/610

ah, thank you. This feels like something that should be supported? How could I make this a feature request for a future version?

I'm looking for a solution to partitioning my single NVME ZFS cache fairly among a few shares. Respecting some type of storage limit on how much of the 2TB NVME cache disk do I want to allow it be used for a given share (but I do not want to block new file writes to the share; just skip the cache before the cache partition becomes technically "full" of data). Not sure if I am making sense but basically I want to set something like this: SHARE1 - store only on nvme_zfs_cache disk (no array). Guaranteed 100GB of disk space. SHARE2 - new files on nvme, then array. I do not want this share to ever use more than 500GB out of my 2TB total nvme cache disk. In short, once SHARE2 hits 500GB used out of the cache - I would expect FUSE to continue handling writes but on the array disks (slower spinning disks). Can I achieve something like this with ZFS properties? It looks like when I have a zfs cache disk that datasets are created for those shares with "cache enabled" so I could set the ZFS quota property bit and force a dataset to have a maximum filesystem size as I want... how would unraid handle this? I hope that it would just go to the array once full (until the mover does its thing later on schedule) root@Tower:~# zfs get all cache/domains NAME PROPERTY VALUE SOURCE cache/domains type filesystem - cache/domains creation Fri Jul 28 22:31 2023 - cache/domains used 96K - cache/domains available 289G - cache/domains referenced 96K - cache/domains compressratio 1.00x - cache/domains mounted yes - cache/domains quota none default cache/domains reservation none default cache/domains recordsize 128K default cache/domains mountpoint /mnt/cache/domains inherited from cache cache/domains sharenfs off default cache/domains checksum on default cache/domains compression off inherited from cache cache/domains atime off inherited from cache cache/domains devices on default cache/domains exec on default cache/domains setuid on default cache/domains readonly off default cache/domains zoned off default cache/domains snapdir hidden default cache/domains aclmode discard default cache/domains aclinherit restricted default cache/domains createtxg 122 - cache/domains canmount on default cache/domains xattr sa inherited from cache cache/domains copies 1 default cache/domains version 5 - cache/domains utf8only on - cache/domains normalization formD - cache/domains casesensitivity sensitive - cache/domains vscan off default cache/domains nbmand off default cache/domains sharesmb off default cache/domains refquota none default cache/domains refreservation none default cache/domains guid 832707413763775451 - cache/domains primarycache all default cache/domains secondarycache all default cache/domains usedbysnapshots 0B - cache/domains usedbydataset 96K - cache/domains usedbychildren 0B - cache/domains usedbyrefreservation 0B - cache/domains logbias latency default cache/domains objsetid 26 - cache/domains dedup off default cache/domains mlslabel none default cache/domains sync standard default cache/domains dnodesize auto inherited from cache cache/domains refcompressratio 1.00x - cache/domains written 96K - cache/domains logicalused 42K - cache/domains logicalreferenced 42K - cache/domains volmode default default cache/domains filesystem_limit none default cache/domains snapshot_limit none default cache/domains filesystem_count none default cache/domains snapshot_count none default cache/domains snapdev hidden default cache/domains acltype posix inherited from cache cache/domains context none default cache/domains fscontext none default cache/domains defcontext none default cache/domains rootcontext none default cache/domains relatime off default cache/domains redundant_metadata all default cache/domains overlay on default cache/domains encryption off default cache/domains keylocation none default cache/domains keyformat none default cache/domains pbkdf2iters 0 default cache/domains special_small_blocks 0 default root@Tower:~# zfs list NAME USED AVAIL REFER MOUNTPOINT alwaysnvme 175M 30.3G 173M /mnt/alwaysnvme cache 10.8M 289G 104K /mnt/cache cache/domains 96K 289G 96K /mnt/cache/domains cache/system 7.87M 289G 7.87M /mnt/cache/system disk1 3.35M 16.2T 104K /mnt/disk1 disk1/appdata 1.85M 16.2T 1.85M /mnt/disk1/appdata disk1/isos 96K 16.2T 96K /mnt/disk1/isos disk2 1.28M 12.6T 96K /mnt/disk2 disk3 1.23M 12.6T 96K /mnt/disk3

Now that unraid support zfs; has anyone found a way to automatically leverage zfs snapshots in the array drives? There are some media files on my unraid server that cannot be played by Plex or transcoded; it seems that I need to use Tdarr to remux MKV to MP4 in order to fix that... I wanted to some kind of ability to "rollback" changes if things don't go as planned. In Synology or other standard truenas this is very easy to do with the snapshots. I am not sure if unraid has any kind of protection at all, other than copying the same file away from unraid before I remux it.

hey would this script automatically show "Shadow Copies" on windows for those unraid shares whose disks are zfs formatted individually (not a zpool of many disks)?

While debugging C states on my other test box I came across an intel C state test tool: https://github.com/intel/S0ixSelftestTool/tree/main - looks interesting. root@ubuntu:/home/ubuntu/S0ixSelftestTool# ./s0ix-selftest-tool.sh -r on ---Check Runtime PC10 Residency during Screen ON---: The system OS Kernel version is: Linux ubuntu 6.2.0-20-generic #20-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr 6 07:48:48 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux modprobe cpufreq_stats failedLoaded 0 prior measurements Cannot load from file /var/cache/powertop/saved_parameters.powertop File will be loaded after taking minimum number of measurement(s) with battery only RAPL device for cpu 0 RAPL Using PowerCap Sysfs : Domain Mask d RAPL device for cpu 0 RAPL Using PowerCap Sysfs : Domain Mask d Devfreq not enabled glob returned GLOB_ABORTED Cannot load from file /var/cache/powertop/saved_parameters.powertop File will be loaded after taking minimum number of measurement(s) with battery only Leaving PowerTOP The system will keep idle for 40 seconds then check runtime PC10 state: The CPU runtime PC10 residency when screen ON: 0.00% The CPU runtime PC8 residency when screen ON: 0.00% Turbostat log: 30.002281 sec CPU%c1 CPU%c6 CPU%c7 GFX%rc6 Pkg%pc2 Pkg%pc3 Pkg%pc6 Pkg%pc7 Pkg%pc8 Pkg%pc9 Pk%pc10 0.82 0.00 98.88 100.03 27.48 68.35 0.00 0.00 0.00 0.00 0.00 0.51 0.00 98.86 100.03 27.48 68.35 0.00 0.00 0.00 0.00 0.00 1.10 1.14 0.00 98.56 0.95 0.73 0.00 99.13 0.61 0.61 0.00 99.05 0.86 0.36 0.00 99.50 0.48 1.70 0.00 98.16 0.83 Your system did not achieve the runtime PC10 state during screen ON Your system Intel graphics DMC FW loaded status is:yes Intel graphics i915 DMC FW is loaded. Will re-check the deeper Package C-state by ignoring PCI Devices LTR value: All the PCI devices LTR values ignore is done! The system will keep idle for 40 seconds then check runtime PC10 state: The CPU runtime PC10 residency when screen ON: 0.00% The CPU runtime PC8 residency when screen ON: 0.00% Turbostat log: 30.003155 sec CPU%c1 CPU%c6 CPU%c7 GFX%rc6 Pkg%pc2 Pkg%pc3 Pkg%pc6 Pkg%pc7 Pkg%pc8 Pkg%pc9 Pk%pc10 0.92 0.00 98.72 100.03 28.93 66.27 0.00 0.00 0.00 0.00 0.00 0.42 0.00 99.19 100.03 28.93 66.27 0.00 0.00 0.00 0.00 0.00 0.78 0.90 0.00 98.69 1.00 0.98 0.00 98.56 1.05 0.62 0.00 98.99 0.89 0.67 0.00 99.07 0.77 1.99 0.00 97.81 0.93 Your system did not achieve the runtime PC10 state during screen ON PCI devices LTR value ignore does not help the PC10, will check PCIe Link PM states: Available bridge device: 0000:00:1d.0 The PCIe bridge link power management state is: 0000:00:1d.0 Link is in L1 The link power management state of PCIe bridge: 0000:00:1d.0 is not expected. which is expected to be L1.1 or L1.2, or user would run this script again. The L1SubCap of the failed 0000:00:1d.0 is: The L1SubCtl1 of the failed 0000:00:1d.0 is: Checking PCI Devices tree diagram: -[0000:00]-+-00.0 Intel Corporation 11th Gen Core Processor Host Bridge/DRAM Registers +-02.0 Intel Corporation TigerLake-H GT1 [UHD Graphics] +-08.0 Intel Corporation GNA Scoring Accelerator module +-0a.0 Intel Corporation Tigerlake Telemetry Aggregator Driver +-0d.0 Intel Corporation Tiger Lake-H Thunderbolt 4 USB Controller +-14.0 Intel Corporation Tiger Lake-H USB 3.2 Gen 2x1 xHCI Host Controller +-14.2 Intel Corporation Tiger Lake-H Shared SRAM +-15.0 Intel Corporation Tiger Lake-H Serial IO I2C Controller #0 +-15.2 Intel Corporation Device 43ea +-15.3 Intel Corporation Device 43eb +-16.0 Intel Corporation Tiger Lake-H Management Engine Interface +-17.0 Intel Corporation Device 43d3 +-19.0 Intel Corporation Device 43ad +-19.1 Intel Corporation Device 43ae +-1d.0-[01]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller +-1e.0 Intel Corporation Device 43a8 +-1e.3 Intel Corporation Device 43ab +-1f.0 Intel Corporation Tiger Lake-H LPC/eSPI Controller +-1f.4 Intel Corporation Tiger Lake-H SMBus Controller \-1f.5 Intel Corporation Tiger Lake-H SPI Controller

Do you mind sharing which specific Kontron motherboard is this?

I have historically downloaded media off torrents; then sonarr/radarr does its thing and moves the files from the download directory to the final media destination for plex. But as you know; you eventually forget about the original .torrent file used to download that media (in case it gets corrupted or what if you want to re-seed). Has anyone configured their unraid setup in a way that: - when you download torrents (sonarr/radarr) you keep a copy of the .torrent file - after sonarr/radar moves the data somewhere; you could decide to keep seeding that media I think because sonarr/radarr move the files and re-organize them, it is no longer possible to "seed" the original .torrent file and other complexities - but maybe I am wrong... I am sure some smart people here are probably doing exactly what I am asking above so looking for some wisdom since I now have unlimited fiber internet and feeling generous with seeding content

Intel NIC cards are picky with the DAC cables and transceivers. I did find a way to unlock the EEPROM by turning on a magic setting hidden in the firmware BUT still DID NOT work for my DAC cable and had to buy a $20 one off amazon. There is a manual way to change the EEPROM hexadecimal setting but this tool supposedly does this for you (I did it manually - go to servethehome forums if you want to find the manual instructions). This tool should do the math and find the right memory address to edit: https://github.com/bibigon812/xl710-unlocker - like I said it did not work for me, but the setting was changed.

I purchased an original intel (non-dell) X710-DA2 network adapter from ebay and quickly came to learn that my SFP+ DAC cables from Amazon don't work with it and that I may need to spend $20 for new DACs... After some digging around I came across some discussions about removing these restrictions by editing the EPROM using this tool: https://github.com/terpstra/xl710-unlocker/blob/master/mypoke.c However I have never messed around with hexadecimal or address records of this type before "00006870 + 00 => 000b" Does anyone have some expertise in this space? is there a better/easier tool to remove these restrictions on intel's latest firmware 9.20? Any crash course on this somewhere? I couldn't find much documented except README.md here: https://github.com/Nevinskas/xl710-unlocker Also someone posted this tool which seems to do the work for you but it did not work for me on 9.20 firmware: https://github.com/bibigon812/xl710-unlocker

Yeah I faced problems with proxmox GUI installer; I had to do CTRL+ALT+F2 to get console then add some forced configuration override to properly detect the iGPU. Here's the guide for the error (embedded framebuffer): https://forum.proxmox.com/threads/generic-solution-when-install-gets-framebuffer-mode-fails.111577/ Speaking of unRAID version, what were you using before? --- Stable was the version I was using previously during my tests. I do confirm, unRAID C10 is unstable with this board I do not know why that is... Proxmox with kernel 6.0 is stable and will 90%+ stick in C10 with no vms running. I made a VM inside proxmox to boot my unraid RC4 usb and it worked, I only played with it for like an hour last night but from my early observations once unraid VM boots and no longer uses resources - the parent proxmox host will keep C10 stable - until you start fiddling around unraid web interface to do stuff then it cycles thru higher C states (but thats expected I think). I'm leaning towards having proxmox as my base OS and unraid as the VM with passthru for the SATA controller (since proxmox OS is installed on 2TB nvme) but will do more testing this evening.

I have achieved C10 (~5 watt) on the Asus Prime Z790M-Plus D4... but there's a catch. I got there by installing Proxmox + kernel 6.xx + powertop 2.15 (from source) + scaling governor set to powersave. root@dom:/# uname -mrs Linux 6.0.19-edge x86_64 root@dom:/# cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/" I wonder if unraid's kernel was newer if we would be able to get to C10 natively as I am seeing its possible here... Also I confirm that Intel X710-DA2 C10 is also achieved under Debian 11/Proxmox kernel 6.0 - the power consumption with 10GB x710 is ~8.7 watt idle. *edit... decided to test rc4 after reading it uses kernel 6.1* results are very close (proxmox is more efficient by 3 watt for some reason - I think C10 is stable in proxmox since unraid is 40% at C10 20% at C8 and random stuff in other states) - 12 watt idle with X710-DA2 connected on unraid 6.12-rc4 root@Tower:~# cat /etc/unraid-version version="6.12.0-rc4.1" root@Tower:~# uname -mrs Linux 6.1.26-Unraid x86_64

@mguttI have a curious question; is it possible to disable HDMI (or virtually unplug) via command line and achieve the same lower power consumption as if I had unplugged the actual HDMI cables from the board? I am planning to install PIKVM https://geekworm.com/products/pikvm-a8 which uses a loopback HDMI connection + USB and allows remote reboot and management of the PC. It would be great if at boot time of unraid i could virtually disable the USB+HDMI connections to drop the consumption but if unraid crashes (and via pikvm I 'reset' the motherboard to get into bios) the video/usb is back for when troubleshooting is necessary. WDYT?

Thank you for confirming that you rarely saw the system go below C8. This is probably as good as it gets with this board, I am happy with it. My other B660M build eats 30 watts idle while this one is 1/3 - same cpu generation too, different chipset and board manufacturer (AsRock vs. Asus) These should be the settings an owner of an Asus Prime Z790M-Plus D4 needs to set to match my power measurements below (unraid usb plugged in + 2TB NVME drive - hdmi + kb/mouse unplugged). I'm still lost on where to set the CPU multiplier in this asus bios (but did set PL1 and PL2 to 35w) - something to look at this weekend I guess when the intel x710 arrives in the mail. *edit note: apparently summary screenshots below are not showing but there are some settings in BIOS with the option "Max battery" and also noticed the scroll window hid my other APM settings where I am forcing L1 on everything plus S4+S5 and CPU deep c-states...

This is a very good pointer... I just received the motherboard + cpu for my new build - without HDDs I am seeing 8.5 watts right now. I partially tried to replicate @riduxd setup (same motherboard) Here's my hardware: MB: Asus Prime Z790M-Plus D4 Version: Rev 1.xx - BIOS version 02/22/2023 v0810 CPU: Intel 13400 CPU cooler: stock intel cooler RAM: 16GB (2x8GB) G.Skill F4-3000C16-16GISB Aegis DDR4 DDR4-3000 CL16-18-18-38 1.35V PSU: RGEEK 12V 300W Pico ATX HDD: only the unraid USB (for now) Idle: ~8.5 watt. - I am not sure if I am reading powertop correctly but this seems to say that I am not hitting C10 at all, and mostly I am on C8? @riduxd any chance you could share what options you have set in BIOS? Just in case I am missing something silly. - Any recommended tutorials on how to force downclock this i5-13400 (65w) to i5-13400T (35w)? Also not sure why at idle my CPU freq is this high... doesn't seem normal to me. root@Tower:~# lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM )' 00:1a.0 PCI bridge: Intel Corporation Device 7a48 (rev 11) (prog-if 00 [Normal decode]) LnkCap: Port #25, Speed 16GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk- 00:1b.0 PCI bridge: Intel Corporation Device 7a40 (rev 11) (prog-if 00 [Normal decode]) LnkCap: Port #17, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk- 00:1c.0 PCI bridge: Intel Corporation Device 7a38 (rev 11) (prog-if 00 [Normal decode]) LnkCap: Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk- 00:1d.0 PCI bridge: Intel Corporation Device 7a30 (rev 11) (prog-if 00 [Normal decode]) LnkCap: Port #9, Speed 16GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk- root@Tower:~#

I'll be buying a X710-DA2 network card soon for my build and I want it to be power efficient. Seems that the X710 meets that requirement. One thing I am not sure about is firmware updates and flashing. Intel has their own flash tool but my research seems to indicate it will not work if your card is from Lenovo / Dell and that you must use their firmware updates - which may lag behind intel by several months / years. I wanted to avoid that PITA if possible. Has anyone crossflashed an intel NIC overriding Dell/Lenovo etc stuff so the card can be updated with the official intel tools?

Thanks for sharing the knowledge @mgutt! As I shop around for an intel lga 1700 motherboard, I found these 8 SATA port motherboard by Biostar but apparently nobody in USA is selling them. Maybe its of interest to some here in other markets: B760MXC PRO 2.0 https://www.biostar.com.tw/app/en/mb/introduction.php?S_ID=1112 B660MXC PRO https://www.biostar.com.tw/app/en/mb/introduction.php?S_ID=1072 I only found B660 board on newegg but seller wants $50 shipping which is 1/4 cost of the board so not worth the risk.

Mind explaining a bit more about how to throttle say a 13600K to 13600T? I thought that Intel blocked undervolting - which I think is one of the things you would do in addition to lowering PL1 and PL2 settings in BIOS. But I think intel just blocked undervolting...