FreakyUnraid

Thanks for the quick reply. That's awesome. Okay, sorry for asking. I Also searched the forum here...

I am planning to move to internal boot. I have bought 2x Intel Optane M10 16GB drives for this purpose. A mirrored internal boot pool. I am watching The Uncast Show's video on this topic before upgrading and this raises some questions https://youtu.be/sruWM1PqjVo?t=495 SpaceInvader One is using a 2TB nvme drive and his screen is showing this: Each bootable pool contains two volumes a system boot pool volume used by unraid a storage pool for general data ... Boot reserved size minimum is 4GB, maximum is 931GB (50% of the smallest selected drive) Q1: What's the point of that storage pool for general data? Does that mean, when using Optane 16GB drives, I will get an 8GB storage pool under Pool Devices? Q2: Why is 50% of the smallest drive the maximum amount the boot partition can be? Why can't we use the entire drive size for this? Just like with USB drives, right? Maybe I am missing something here...

Well, I am absolutely not an expert but also not a complete noob so I'll stop arguing about the logic behind this. Fact is, it causes issues. I believe Unraid has to do a better job of explaining these settings to users. I followed Spaceinvader One's video's when setting up my server. This is the setting for TV Shows usershare. My current settings just seem to break Mover. Mover doesn't even move on to another drive. It keeps trying to move the same (.partial) file to this drive for a couple of hours now... And the log gets filled with "(28) No space left on device". Mover seems to not be able to handle this situation? This doesn't seem user friendly at all... I have to find this out by going in to the logs and see what's going on. No notification. I then have to manually, in console (also not user friendly), stop mover and now go use a third party plugin to empty out my drive before I can continue Mover. I really would like to hear what the settings for a TV Shows usershare should be, because to my recollection all youtube video's recommend the same split level settings? I kind of understand the problem you describe, but I can also give some counter arguments to this with my noob level of knowledge, but I'll just shut up now. Well that was basically my thinking with doing this. Most files are max ~25GB big. So with setting it to 50GB I would have enough space left and if a file goes over this I would move things around manually to balance things out a bit more. Currently this "free space" setting only seems to be for my cache drive? If the cache drive hits that limit, then it will start to directly write to the array... Like why is this setting also applied to the cache? Fill them up 100%? So no data can be changed at all in the future, like metadata? Performance of the drive, or at least for some data, down the drain? Don't think that is smart. And what about the new options for SSD pools? Isn't the advice to only fill SSD's to max 80% capacity? Like I said, I followed Spaceinvader One's guides. I have this set for the TV shows share and I have seen several other tutorials that recommend the same setting for such a share. Also, the explanation in the GUI doesn't mention anything like this at all. It just explains how it will manage the creation of certain directories, but not that you should keep an eye on the drives and manually move things around when using these settings... And again, this really seems like weird design. Mover can't even handle this split level situation? Don't worry, this is just my single ssd download cache. Appdata has it's own dedicated zfs pool. Because the mover tuner plugin went bad I have since then used the logic of just filling the download cache to ~90% and then manually hit mover. I have set notifications when the cache gets filled beyond 85% and then I go in and run mover over night manually before going to bed.

I understand split level somewhat , but I don't understand why this gets priority over the set amount of free space. If no free space is set and the drive is full then mover will move to another drive, correct? If I have a 2TB drive and want 100GB free space as a minimum, then mover should just see this as a 1.9TB drive and move on when moving a file goes over that threshold. Just like it would when a drive is full. Ignoring the set free space seems so illogical to me? I now end up with drives that only have 2GB of free space left... How does everyone else handle this? Is this some old logic left from the early days? I can't imagine you would want this bahaviour when you don't use normal HDD's, but only SSD's? With the current addition of pools etc. I mean, those don't really like it when you go over 80% or something, right? But I'm getting distracted here. Question I have now: How do I configure Unraid to get it to respect the set amount of free space? Is this even possible or do I have to use Unbalance every time manually? This is mostly *arr media files, so folders are not created by me manually.

@itimpi I have always used this setting: Automatically split only the top two directory levels as required Why does Split level not respect Minimum Free Space setting? I don't understand why this just ignores "keep this amount of space free". If the disk is full it goes on to the next disk. So why not "oh keep 50GB free, then disk is full now, let's move on". What's the logic here?

No, I stopped using that plugin after upgrading to Unraid 7.0.0. Mover was completely broken at that point. I tried it once again like a month later, but still had problems. I'm currently using mover manually when I see my cache drive is full.

For all shares I have set the following: Minimum free space = 50GB Allocation Method = High-Water Problem: Mover fills array disks to 100%. Drives are filled with only 2GB left... The files that are moved are way less than 50GB, so I don't understand why mover keeps writing when a drive hits that 50GB threshold. I don't want mover to fill my disks to 100%. I want to leave some free space. And I choose 50GB because Minimum Free Space also applies to the cache drive... I want to be able to fill my 1TB cache drive

Okay, perhaps my understanding of writing corrections to parity is not 100% correct. This reinforces my argument for adding more information / context for the user. But even so, I don't see a reason to immediately write corrections to parity without giving the user the ability to first check what was found by parity check.

Writing corrections should be OFF by default. Check: A process of examining or verifying something to ensure it is correct, consistent, or functioning as expected. The system says: Parity is valid. The system does a check if that assumption is still valid. A parity check stresses your hardware like no other operation. Cables, HBA/Sata controllers etc. If those somehow fail during the check, the "write corrections to parity" will ruin your parity. There should always be a check first. A second run should be used to write correction AFTER user intervention. This gives the user the ability to check the logs, see what happened and if there could be a hardware explanation. If so, fix that and do another check before you alter / ruin your parity. "You can always do another correcting parity check if the first one was because of hardware failure"; Yes, but you assume no HDD will die during that second process. If that happens, you can say bye bye to your data. In all settings there is an explanation of what a certain settings does. On the Main Tab there is no explanation under "Check" or "Parity Valid", just that it will start a Parity-Check. It also does not explain what "write corrections to parity" precisely mean and what the possible dangers of this setting is. My suggestion would be to add more information here. And possible move the question to write corrections to parity to a pop up window that will specifically ask you (and explain) if you want to write corrections to parity. If it is the first parity check since / parity is valid it should suggest the user to just do a check, without writing corrections.

I would like an option to disable the mover schedule. A user shouldn't be forced to pick a schedule. Choose a mover schedule ranging from hourly, daily, weekly and monthly. The interval determines how fast the mover will activated, it runs in the background. Setting a fixed schedule does not work for me. My downloads and work are to irregular for a fixed schedule. You might say 'just use the mover tuner plugin'. Right, I don't think this basic feature should be a plugin. Second of all I am running Unraid 7.1.2 and unfortunately the mover tuner plugin has been unreliable since 7.0.0 and I don't want to deal with that anymore. And although Mover "runs in the background" it will take up a lot of my cpu power and it will take a long time when my 1TB cache is almost full. So I don't want to have it run when I'm downloading / seeding or doing other stuff. I have set up notifications that will notify me when the cache is 80% full or higher. That way I know when I need to run mover; "cool, I'll stop my work tonight, stop downloads and the *arr apps (because of their "scan library task") tonight and let mover do it's thing". Somewhere in the morning it's done and I can resume everything again. That way I am in control. Now I had set it to "monthly" and this morning I was trying to do things and just losing my mind why my server was so slow. 1st of the month... mover is still running...

Yes, well, there is no warning before running mover so I didn't know that before. If I had known it would take mover 1 month to complete... I just went with what I had learned previously with changing cache drives. Move appdata to array. Change drive and put everything back. Before trying different things I would like to know how to stop the moving process. mover stop mover: not running While I see in the Main tab read/write on both the array and btrfs pool at the same time and at the same speed, telling me that mover is still doing it's thing. Only thing I can think off is that it's finishing it's 'cache' of files it had already lined up for processing? Anyhow, it's been 3 full hours and it's still going and I have no idea how to monitor what's going on or how to safely stop it, aside from stopping the array or rebooting the server. EDIT (before posting) and for anyone getting into this same situation; I had a chat with chatgpt and checked everything afterwards with Gemini to make sure Told it what I did and what was happening right now and if we could try the least invasive options first to fix this. It wanted me to look which processes were busy with files on my btrfs pool. lsof | grep "/mnt/cache_raid" This returned a lot of the same UID 122 messages, but also a couple of 'find' messages lsof: no pwd entry for UID 122 find 3707572 root 6r DIR 0,44 206 856278 /mnt/cache_raid/nextcloud/appdata_ocwgts04pj90/preview/8/8/a/c/7/2/8/371816 chatgpt suggested next to kill the process with "kill -9 3707572", but after googling "-9" (terminate process immediately) and being afraid of data getting corrupted, I opted for for "kill 3707572". After running that it took around 15 seconds for the Main tab showing me that read/write stopped completely. Looking at syslog straight after that it showed this: Jan 21 15:17:48 Server root: ionice -c 2 -n 0 nice -n 0 /usr/local/sbin/mover.old stop Could this perhaps have something to do with the mover tuning plugin and Unraid 7? I'm seeing mover and mover.old script. After the upgrade to Unraid 7 mover wouldn't move. I googled and removed the plugin after reading that I had still an old version installed. I thought Unraid would warn users for incompatible plugins before upgrading? Anyway, after removing that plugin and rebooting the server Mover was working again. But I wonder why mover.old is now the cause of this all... Or maybe was I wrong for killing the process first and should I have just ran "mover stop"... The mover part has been fixed, so now I need an option to move all the small files to another drive. @JorgeB Do you mean with Midnight Commander I should use Krusader? Just move files from one window to another, into the right disk/folder? Chatgpt suggested rsync rsync -av --ignore-existing /mnt/cache_raid/<sharename> /mnt/user/<sharename> I was using a specific disk in my array, so maybe change out /mnt/user for /mnt/disk ? And perhaps I should just use an unassigned drive for this, so the array won't slow things down doing parity calculations? Given the fact there are so many small files. I will be deleting that preview folder for sure and just redo that when it's all back up. Sorry for the long post.

I'm having an issue where mover is extremely slow with moving a lot of small files. And when trying to Stop Mover the process just continues. Even though the process is killed and the GUI says that mover isn't running. Current build: Unraid 7.0.0 1x 1TB NVME = cache drive & appdata folder (appdata = 90 GB) 2x 2TB SSD BTRFS pool = nextcloud & plex folder (1.18 TB) I want to re-organize and upgrade. For this reason I started with emptying the BTRFS pool to the array. I changed the mover location in Shares for just the Nextcloud share and started the mover. Overnight the used space on the pool went from 1.21 TB to now 1.18 TB... That's almost 12 hours for not a lot of data. I understand that all these shares contain almost nothing but small files and reading online I can see that mover has trouble with these kind of situations. So, in order to try and find a different and faster solution, I stopped the mover. Kill PID > this killed the process, but files are still being moved. Stop mover > this updated the GUI, but files are still being moved. So mover isn't moving according to Unraid, but files are still being moved just looking at the read/write action on the Main tab and it's been over an hour now since issuing these commands. I would like to move forward, but I don't know how. 1) What's the best course of action to really stop the moving of files? Stopping the array? Rebooting the server? I haven't done that because I'm afraid of damaging files that are in transit. 2) While I'm here asking for help; What is the best practice of emptying both drive (pools) and start over? It's a lot of data with mostly just small files. For context; the above setup is going to be transformed to: 1x 1TB NVME = cache only 2x 500GB SSD ZFS pool = appdata (including Plex) 3x 2TB SSD ZFS pool = personal files (including Nextcloud share)

Are you sure about that? These 2,5" Seagates only do about 135MBps max. HBA has 8x 6 Gbps With 1 link to the expander; that's 4x 6 Gbps = 24 Gb With max 20 drives that comes to 24/20 = 1,2 Gb per drive = 1,2 / 8 = 150MBps PCIe 2.0 does 500 MBps per lane. HBA is x8 with 2 ports which comes to x4 per port? So 500 MBps x 4 = 2000 MBps / 20 drives = 100 MBps max theoretical. In practice this will be lower is my guess. So PCIe 2.0 is the bottleneck here. Changing out the PCIe 2.0 card for a PCIe 3.0 card result in: PCIe 3.0 does 985MBps per lane x 4 = 3940 MBps / 20 drives = 197 MBps I read that in practice this is more like 3200 MBps / 20 drives = 160 MBps Still well over the max these drives can do. Around 20% headroom. If, in the future, I would switch to 3,5" drives which can do much higher speeds I will also need fewer drives. So I think I'm good when I switch to something like a 9207-8i, right? The 2,5" drives need more HBA/Expanders so the power consumption difference between a 3,5" setup is indeed negligible at this point. I just really like these little drives for being so quiet. Even in 100% operation (all drives) I can barely hear it. I remember my old Synology with 4x 3,5" drives which was in the same spot the current server is, and the noise was really unbearable. The power consumption 'demand' was for the new hardware. Power is getting expensive so I don't want a HBA or Expander that consumes a ton of power when there are (far) better and more efficient once out there.

Hi, Currently running a fujitsu D3643-H (4x sata) with 2x IBM M1015 (2x 8 sata). All ports are populated with 2,5" drives. With 20 drives I'm at capacity and I need more storage. No more ports left, so looking for a good solution to expand the number of ports. Less money and less power consumption is much better. Options I considered but deemed not a good option: - Changing HDDs for larger ones. Already running 5TB drives, so can't go bigger in 2,5". Going 3,5" would mean I have to change out at least 3 drives (I have dual parity) to go bigger. With at least 16TB being the best option this would set me back at least $800. Too expensive and also 3,5" make too much noise for my taste. Server is located in a room I often work and sit in so it's a big deal. The 2,5" are so quiet, I can't even hear them. And I have a couple spares laying around. - Changing 1x M1015 for a 16 port HBA. Again, this would cost a lot. You can get one for around $170 on ebay, but I would like to get 2 so I have a spare laying around just in case. Ebay and shipping can take weeks if not months and I don't want my server to be down so long. With $340 this would be too costly in my opinion. The only real option I found was to change out 1x M1015 for an expander. Looking around the Intel RES2SV240 looks to be the best option? For around $60 not a bad deal? The Lenovo is real cheap at just $30 but I think I can't get enough ports with that option? After reading the performance topic on throughput I'm a bit worried that the PCIe 2.0 M1015 is going to bottleneck my drives quite a bit. They start parity at around 135MBps. With (in theory) 20 drives on the expander this would mean they would be bottlenecked to around 113MBps if my math is correct? Could this be damaging in some other way than just parity taking a bit longer? Perhaps I should change out the M1015 for a PCIe 3.0 card like the 9207-8i and sell my M1015's? So in short: NOW 2x M1015 Option 1 1x M1015 (PCIe 2.0 bottleneck with 20 drives on expander?) 1x Intel RES2SV240 Option 2 1x 9207-8i PCIe 3.0 (Dell or a different one?) 1x Intel RES2SV240 Option 3 ? Suggestions are welcome. Would love to hear your thoughts on this. Thanks.

For people experiencing the same issue: delugevpn is running but the Web UI is not available. Edit the container and check that LAN_NETWORK is set to your local LAN IP (192.168.1.0/24 for example). Mine was set to 'localhost', which resulted in DelugeVPN and all dockers routed through it not being accessible. All Web UI's were unreachable. I'm not sure how this happened, because until yesterday everything was working just fine. And I followed Space Invaderone's video when setting things up and I re-watched that again and he also put's in the LAN IP. So I have no idea where 'localhost' came from... Could an update of the container have caused this?