Faceman

The chipset has a PCIE switch included in it, which behaves similarly to a network switch for PCIE data, so they just share bandwidth as needed and only bottleneck when you try to hit them all at once while doing a lot of other things (other pcie slots, sata ports, usb networking etc) but the chipset is pretty smart in how it shares the bandwidth so it really isn't an issue in normal use. Your DMI4.0 x8 link from CPU to chipset has around ~16GigaBytes/s of throughput, ~128gigabit/s. so you would have to put 3x of the fastest PCIE 4.0 x4 SSDs in there and run them all simultaneously at nearly full load for it to throttle, and even then they will be more than fast enough. if you used a perfectly good mid spec SSD like a WD Blue which do around 5GB/s, you could run all three without noticeable bottlenecks as long as the chipset isnt doing much else at the time.

I don't see this as a bug. it's been a feature of OpenZFS for a few versions now but it seems like it has either not worked in the past or the underlying behavior has changed and now more people are noticing it. I remember when I first switched to ZFS for my cache pools in 6.12 the ARC usage sat at the allocated max permanently, but now it does scale up and down with usage. If I fire up something busy hitting my ZFS pools with lots of repeated reads and writes the arc usage quickly maxes out and stays there, but if I let my system idle for a while it dials back down to 4-6gb out of my 16gb allocation and if I stop all containers and really drop usage to zero the arc usage pulls back to around 2gb, which is the currently set zfs_arc_min of 1/32 of system ram (I have 64gb on this system) I think it works well, but I haven't been able to test setting zfs_arc_min which we are supposed to be able to do according to openZFS.

supposedly it should support being set separately with the max: options zfs zfs_arc_min=8000000000 options zfs zfs_arc_max=16000000000 for 8gb min and 16gb max for example, and some have set it to the same as the max which would theoretically fix it to full. but might be somewhat broken in openZFS still? I'll test it next reboot, i'm seeing mixed reports on that but there are a lot of different versions and different platforms.

if the shares are set to "array" only the mover wont do anything, even though there are files in /mnt/cache/share it ignores them. You need to set the shares to Cache>Array, so set primary as cache, secondary as the array, then the mover action to cache>array, then the mover knows it needs to clear the cache. now when you force run the mover it will notice there are files to be moved. You CAN move them manually, but you need to ensure everything goes to the right spot and you are writing INTO the folders and not overwriting them, and you need to maintain permissions. I would recommend not using --remove-source-files until you have copied everything and verified it is all good then format the disk. just in case something goes wrong.

Only method I know is to reformat all of the drives and start over. Maybe someone smarter than me has a less destructive way to do it?

FYI, LSBLK is showing 4 partitions on each data disk, two smaller ones and then a 4tb and 2tb partition on those data disks. The parity disk has only a single 6tb partition which is correct, but the data disks partition tables were probably leftover from Synology.

4x10g wont immediately make your normal, single client SMB transfers faster than 1x10gbit, but hitting the server with multiple streams at once will be faster, if you have multiple 10gbe or faster clients hitting the server then you will definitely see improvements under load. there is some tuning you can do for SMB multichannel and other SMB3 optimizations to further improve it if you have a client PC connected with 4x10gbe as well, but it's never going to be perfect other than for specific, parallel transfers.. that's part of the reason I went with 25gbe instead of 40gbe, it's easier since it is a single 25gbe channel, not 4x10gbe channels you need to distribute the load over.

Could it be related to this? issue affecting other intel Alder lake-N platforms?

you don't want to be dual booting on the file server, your data would only be available properly when the NAS OS is running and you run the risk of damaging something on the NAS side while windows is running or vice versa. You want to put a NAS OS on the server and put it away somewhere then only access it from your other machines. Running your main PC as a VM within unraid or another OS isnt too hard if all you need is a basic desktop for file management and light apps, but there are some caveats and incompatibilities that can crop up and make things hard for beginners, especially with GPUs and software compatibility under VMs, I'd recommend a second system entirely for that. Keep the NAS as just a storage and applications server, don't try to use it as your main PC too. Keep researching NAS operating systems and try some out with some test data, Unraid is fairly simple but most of the people here are long term users that might not understand the difficulties of coming in as a new user with limited experience or existing knowledge. Truenas is another one, there is a free version but it can be more complex to configure than Unraid without some prior knowledge. hexOS is a newer one that is also paid and a bit expensive in my opinion, but it's mostly just a simpler UI on top of truenas. OMV might be up your alley as it is pretty simple, but it's imperfect and lacks some features. Unraid is very flexible and feature rich, it does let you bend it to do whatever you want, but that can result in feature overload if you don't know what you are looking at. All of these options will be better than an off the shelf NAS with proprietary software though, despite the learning curve. All of that said, you can use windows or linux as a files and media server while using it as a desktop too, they are just a bit clunky compared to a dedicated NAS OS.

This has hit me before and i had to replace 8TB of content, the important parts I had on my backup but some was lost. I believe the wording in that menu option needs to be updated.

Are your switch ports configured as 802.3ad? sometimes there are other methods, but LACP (802.3ad) is the most common and well supported. I used to use a 4x1gb bond on unraid without issues to feed 6x1gbe clients with good simultaneous throughput and it worked perfectly

I find sometimes the CTIME gets changed by something or doesn't make sense upon creation sometimes.

Have you got a screenshot of your plugin settings, first thing to check and easy to miss a setting and not realise it.

I turned off mover with the "Disable Mover running on a schedule" option a few days ago to help speed up a parity operation, but when I came back to it and looked at the logs, it kept running as normal, on schedule. so there is definitely something up with that option. Just to test, I just turned it off again, and sure enough it ran again the next time my schedule came around.

try removing the device mapping and adding it via the Extra Parameters "--device=/dev/dri:/dev/dri"

Disable filename validation in the Mover Tuning settings to skip the filename validation checks. $ and other special characters in filenames cause all sorts of havoc in Bash scripts and are best avoided if possible, but it is actively talked about in the plugin support thread.

A ZFS pool is a cache pool, so if you want a share to use it, you need to set up a primary/secondary under the mover settings and use it as a temporary cache, so when the mover runs it dumps the entirety of the shares contents on the ZFS pool to the array and you start filling it up again. since you have a cache and a ZFS pool you are a bit stuck as Unraid currently has no 3 tier moving functionality, only primary (new files written here) and secondary (once the mover runs it stores data here). There is no easy way to have a share spread across both the array and a pool seamlessly as if they were one larger pool. You can move the contents of a share on one disk to the pool manually, but if the share is set up to use a primary and secondary the mover will just move them back again. If you want to use both somewhat automatically for a media share, look at setting up the mover tuning plug-in, that would let you have your new media live on the ZFS pool (set that as primary) and only the oldest files get moved off the ZFS pool to the array when space is needed, slowly tricking the oldest files off. that is how I run my setup, when the ZFS pool exceeds 75% full it moves the oldest files to the main array until it hits 60% full then fills up again.

We've been asking for even just the most basic mover tuning features to be worked in. I know Limetech like to keep the base OS as simple as possible for many, many reasons (native ZFS and WIFI are huge steps for limetech!) but this is so essential to a good high performance server setup I feel it justifies the complexity. many features started out as plugins or addons, even the webui started as a plugin.

Can you post a diagnostics file? Really cant recommend trying to use SSDs in the array though, they should be in a pool.

I used to find disks that way, but stopped after a similar incident of disks dropping out unexpectedly just after mounting, just dirty connectors and a confused controller thankfully. I did pull the backplane out at one point and hit every connector with contact cleaner spray just in case, some of them were surprisingly dusty, obviously empty for years. Now I have a map of my disks and which bays they are in, but also when the array is offline I spun all disks down, then spin up just the disk I want to pull and look at the activity LEDs on my enclosure to confirm before yanking the disk out. I also never pull a disk with the array started, even if it's not part of the system at all, just in case the bending/flexing of the backplane causes an active disk to have issues.

Check your mover tuning settings for the clear empty folders options, they have been a bit broken lately and are still being worked on. Unfortunately it cant go back and clear empty folders if they were already empty, it will only clear an empty folder if it actually moved all the files out of it. Make sure you are on the latest version of the plugin and the settings are configured to clean empty folders.

if that's the whole error, and its dropped out the rest of the filename after the #015 then it's not handling the # in the filename properly. There is nothing technically stopping you from using a # in a filename (honeslty not sure about folders, likely the same), but scripts need to be carefully written to handle them properly. Best to avoid them if at all possible.

Unraid can fail over to emulating a disk live, but could it also trigger a rebuild live too? I'm imagining a perfect world where you can mark a disk for replacement, unraid drops it offline to emulated mode, you pull the disk and swap it with a new disk and it just keeps on ticking and rebuilds in the background.

What are your settings for mover tuning, and if those shares have custom per-share settings?

Pretty sure that's just the containers own /dev/shm, the /transcode is there and the fact that it says 16G means its likely correctly mapped to the hosts /dev/shm, assuming you have 32gb ram? Then in your plex transcode settings you have /transcode as the temp directory, so it should be working.