Jump to content
Arcaeus

How does Unraid spread files across the disks?

52 posts in this topic Last Reply

Recommended Posts

So I've just set up my Unraid server and got it running (with a HUGE thanks to @Squid @Hoopster @Frank1940 and others). I started with 2x 4TB (one parity and one data) and a 256 GB 850 Pro SSD as cache from an older computer. After I get everything up and running, I cleared off another 2TB and added it into the array.

 

I'm looking through my drives and see that the 4TB data drive is 75%ish full, and the 2TB I added afterwords is basically empty. So I was wondering, how does Unraid move/spread data across the drives? Is that what the Mover does, or does that just transfer from cache to Array?

 

And when I add another drive later on, is there a process I should do to make sure that it's spreading the data evenly? Or any setup tips for that kind of thing.

Share this post


Link to post

Your question is about Allocation Method. High-water is the default and is a good compromise between spreading the data without constantly switching disks.

 

Once the 4TB gets to 1TB free it should start on the 2TB.

 

There isn't really any good general case reason to spread the data evenly, or really even a good definition of what "evenly" would mean in the general case where drives can have different sizes.

 

Here is the wiki on Allocation Method:

 

https://wiki.unraid.net/Un-Official_UnRAID_Manual#Allocation_method

Share this post


Link to post

Make sure the setting for that share directory is set to use ALL disks. Default when adding a share is ALL, but if you might of changed it messing around in the beginning theres a chance it is set to use Disk 1 only. Doesnt hurt to double check. It wont start filling the 2TB unless its told to use that disk.

Share this post


Link to post

@sminker I went through and checked all of the shares, and they are all set to use ALL disks.

 

It looks like it started to move some stuff over, but then stopped after it got past the 1TB mark. Based on the guide @trurl sent over, it seemed like it would make multiple passes until the data was spread relatively "evenly" across the disks.

 

Also, my cache drive keeps filling up. I've started Mover manually to get that off of there, but it doesn't seem to be completing the move. It will get down to like 178GB or so and then just stop. Which then fills up again and starts giving me all of these warning messages.

 

image.thumb.png.41c0d3d3604fa73604d87eb8d052b22d.png

Share this post


Link to post
5 minutes ago, Arcaeus said:

Also, my cache drive keeps filling up. I've started Mover manually to get that off of there, but it doesn't seem to be completing the move. It will get down to like 178GB or so and then just stop. Which then fills up again and starts giving me all of these warning messages.

 

image.thumb.png.41c0d3d3604fa73604d87eb8d052b22d.png

Do you have any shares that have files on the cache that are NOT set to Use Cache=Yes?   Likely candidates would be the appdata or system shares which by default are set to stay on the cache so you get good performance from docker containers and VMs.    Also any open files will not get moved.

 

If none of these explain what you are seeing then I would suggest that you enable move logging and then start Mover manually.  When it completes get the system diagnostics ZIP file (via Tools->Diagnostics) and upload it here to see if anyone can spot the reason for the behaviour you are seeing.

Share this post


Link to post

My appdata share is set to Prefer, but that shouldn't take up 170-some-odd-GB, right? I only have like 6 dockers on there at the moment (Plex, Emby, Sonarr, SABnzbd, Krusader, Deluge) so my guess is that there are some torrent files that aren't being moved over properly.

 

And where is move logging located again? If I remember correctly, it was on the Main tab with all of the Mover settings. Which for me is not displaying at all.

 

image.thumb.png.a801ebf23af98b9a800fa72f8477ee58.png

Edited by Arcaeus

Share this post


Link to post

Appdata could easily be that large as Plex can be a bit of a space hog.   It might be worth trying to get the size of the various folders under appdata to see what space each app is using.

 

the mover logger setting is under Settings->Scheduler.

Share this post


Link to post
1 hour ago, Arcaeus said:

there are some torrent files that aren't being moved

If you are seeding then you would probably have to shut down Deluge and run mover. I have all my downloads go to a cache-no share.

 

My (admittedly modest) plex library is about 4G in appdata.

Share this post


Link to post

Video preview thumbnails in Plex can occupy a huge amount of storage so you might want to make sure they are switched off.

Share this post


Link to post

You can browse the cache to see whats on there. Do a manual Move operation. After its done click on the little file button on the far right of the cache drive from the "Main" tab. There should really only be appdata, domains and system on there. If theres more go that share, set it to "yes" for cache. Invoke the mover again and it should move it off the cache drive. Once complete change the share back to "No".

Share this post


Link to post

@sminker Ok great I'll try that. However, when I click on the "Move Now" button, it shows the "Disabled - Mover is Running" text. I refresh the page, and the button is lit up again like it's not actually doing anything. If I remember correctly before, I could switch tabs and come back to the Main tab, and if the operation was running, it would still be grayed out.

 

I tried it yesterday, and no data seems to be moved off of the drive.

Share this post


Link to post

Did you turn on mover logging or stop seeding as suggested?

Share this post


Link to post
Just now, Arcaeus said:

Mover logging is on

So what is in the log then? Post diagnostics

Share this post


Link to post

Cache "prefer" tries to move the data to the cache drive, which is full and not what you want, change that share to cache "yes", turn on the help for info on the various options.

Share this post


Link to post

It is trying to move appdata to cache, which would be correct except for the fact that it is already full. We need to work with each user share to try to get things except appdata and system moved off cache to make room for those things that belong there. Give me a bit to look it over some more.

Share this post


Link to post

Go to the Shares page and click the Compute All button, then wait until it completes (may take a while) and post a screenshot.

Share this post


Link to post

@johnnie.black Ahh ok, I misunderstood that. Looks like Mover is running now so we'll see how far that gets. 

 

@trurl Alright, that is processing, will post results on completion.

Share this post


Link to post

The diags are a bit confusing since it says Downloads has default settings, which means it has never had any settings made and so would be cache-no. But the logs say it is trying to move it to cache. And you have another .cfg file in there that looks like it might be a duplicate of Downloads.cfg. It is cache-prefer but diags says it has no files.

 

I know there was a problem with share names not conforming to SMB and duplicates could get created when you used the wrong upper/lower case (probably what happened here) or an illegal character.

 

Go to the settings for the Downloads share and make sure it is set to cache-yes so mover can move it off cache.

Share this post


Link to post

Ok, went in there and confirmed that it is set to cache = yes.

 

Mover is still running but after 15 minutes I don't see any change in the used/free space. Shouldn't moving files be a straightforward process? Or is it slower due to a parity check/multiple drives/etc?

Edited by Arcaeus

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.