Hardware:
Dell R730XD

Dual Xeon E5-2650 V4

SAS Cards: Dell H330 Mini and LSI 9305-16E

2x18TB Seagate Exos Sas, 3x16TB Seagate Exos Sata, 12x8TB HGST Sas

128GB ECC memory

Dell Quadro P4000

 

Plugins:

Appdata Backup 2024.11.21
CA Mover Tuning 2023.12.19
Community Applications 2024.11.21
Dynamix Active Streams 2023.02.19    
Dynamix File Manager 2023.06.01
Dynamix System Temperature 2024.07.10    
File Activity 2024.07.17
Spin Down SAS Drives 2024.02.18
Appdata Backup 2024.11.21    
CA Mover Tuning 2023.12.19    
Community Applications 2024.11.21    
Dynamix Active Streams 2023.02.19
Dynamix File Manager 2023.06.01    
Dynamix System Temperature 2024.07.10
Fix Common Problems 2024.11.12
FolderView 2024.10.02
GPU Statistics 2024.11.18
NerdTools 2024.02.17
Network UPS Tools (NUT) for UNRAID 2024.10.17    
Nvidia Driver 2024.07.10
Tips and Tweaks 2024.11.17a
Unassigned Devices 2024.11.16    
Unassigned Devices Plus 2024.04.01
Unassigned Devices Preclear 2024.11.12
unbalanced 2024.09.10
Unraid Connect 2024.09.11.1029
User Scripts 2024.03.29

 

Problem:

 

A few months back, I had noticed that when I initiated mover or ran on a set time schedule, files moved onto the array would only be put onto Disks 1,2, and 3. This was around the time that I had installed 3 new SATA drives (I did a rebuild for each individual drive I believe, swapping 1 drive at a time until all 3 were installed) as well as adding in a LSI 9305-16E to run a SuperMicro 847-36 Dual SAS3 backplane enclosure. It had taken some time to fill up the drives that were added, but when they were, I noticed that the SATA drives were the only drives in the array to have files moved to with mover. With this, drives 2 and 3 would fill up to half and then drive 1 would fill up to full until a error was thrown about drive utilization being at max and not fill up other drives. I can move files from drive 1 with krusader/unbalanced no problem to all the other drives in the array, which is the 8TB drives so I know they work and I am getting no errors with any of my docker programs and needing to read from the array after the manual move. When this started happening, I did not have the time to research it until now with the holidays and actually having time off work.

 

This was also around the time that I had setup Nextcloud, setup its own share and only have it on Disk 9 of the array to consolidate the upload directory for the docker. This might be the problem as it is only on one disk in the array, but I have not gotten that far in troubleshooting this.

 

What I have done to try and fix:

 

1.  Updated unraid to the latest version, 6.12.8 > 6.12.13 to see if it might have been a bug. This did not fix the problem.

2.  Move the SATA disks off of my R730XD backplane to one of the SuperMicro 847 backplanes to see if isolating them would fix the problem, which it did not.

3.  Go into the shares and set them from High-Water > Fill-Up > High-Water just to make sure that the allocation itself did not get screwed up with the disk changes a few months back.

 

Logs are attached, but from what I could tell reading through it (With my limited knowledge), and did see some mdcmd errors, but was not sure if this was related to the problem or not as it looked like these came in after reboot when the array is not started.

 

Just trying to figure out why it is not following the High-Water algorithm when there are other drives in the array that are empty and can take the files. Maybe its because the new drives are double the size of the old ones? Not sure, which is why I decided to post here in the support section to try to get more eyes on this and see if I can fix the problem without switching to Most Free and see if that works (But if I need to do this to see if it will work, I will).

 

Edit1: Added plugins because I forgot to add them with their version dates.

main-diagnostics-20241126-0724.zip

Edited November 26, 20241 yr by firepong

With high water and mismatched drives.

 

System will start at disk1 and fill it to 7.5T free, then move to disk 2 until its at 7.5T then disk 3->7.5T

 

Following that it will start again at disk 1 and fill to  3.75T, then disk2 -> 3.75T, then d3 ->3.75T

 

After those drives are done it will fill disks 4->9 to 3.75T then begins again at disk 1 and fills 1-9 to 1.875T and so on.

 

3 minutes ago, Squid said:

With high water and mismatched drives.

 

System will start at disk1 and fill it to 7.5T free, then move to disk 2 until its at 7.5T then disk 3->7.5T

 

Following that it will start again at disk 1 and fill to  3.75T, then disk2 -> 3.75T, then d3 ->3.75T

 

After those drives are done it will fill disks 4->9 to 3.75T then begins again at disk 1 and fills 1-9 to 1.875T and so on.

 

So what your saying is that it will fill the 16TB drives first before starting on the 8TB ones? If so, I just learned something new today. Been using Unraid for years and never knew this was a thing

 

If the above is exactly what you are saying, then I will mark this solved and just leave it as is. With the miss-match, would Most free be the best use-case for miss-matched drives to ensure it fills all drives evenly instead or just say screw it and let it fill the 16tb drives before moving to the 8TB ones?

Edited November 26, 20241 yr by firepong

I would just say screw it.  But the choice is yours.  The mismatched thing isn't an issue, but a common misconception that it means 50% of the drives.

Just now, Squid said:

I would just say screw it.  But the choice is yours.  The mismatched thing isn't an issue, but a common misconception that it means 50% of the drives.

 

Got it. Ill just live with it and let the drives fill up. 👍