Shares not creating folders in disks

[morallydubious]

Hi,

 

unraid newbie here so please forgive any faux pas. My Google fu has failed me on this occasion and if I am the only one having this problem then the likelihood is I have done something uniquely wrong.

 

I have a 5 disk setup + cache drive. 1 8tb parity, 1 8tb data + 4 4tb data. The two 8tb disks were new and used to create the array (1 parity + 1 data). Shares from my old freenas server where then mounted and the data copied over to a ‘transfer’ share on the unraid server. I then created a temporary Plex docker to test the mainly media files that had been copied over. So far I am happy.

 

Next I add the disks from the old freenas server and the cache drive. Great, so now I have a 91% 8tb data disk and 16tb of empty disk.

 

I want to distribute my media files because of the possible number of Plex connections, so my idea was to create shares for movies, tv and music including all disks, most free, split level 2. When I created the shares I noted that the folders for the shares where only created on two of the disks. Puzzled I attempted to copy all tv shows (using Krusader) from transfer share into the new tv share and notes that data was copied only to the 2 disks with the folders created and very unevenly towards the first disk.

 

I am running the trial, is there a limitation? I am passed the point of no return anyway so I am going to purchase if a can resolve this.

 

Thanks in advance

 

 

[trurl]

Split level has precedence so if split says files should go together then that is where they go. 

[morallydubious]

Thanks for the response turl but I am not sure what point you’re making.

 

My assumption was that with a split level of 2 for share tv I should see the following on creation:

   Disk 1

      tv

   Disk 2

      tv

   Disk 3

      tv

   Disk 4

      tv

   Disk 5

      tv

 

Given the other setting specified above I would also expect each ‘show’ folder to be created on all disks then ‘seasons’ to be restricted to 1 disk.

 

What I actually saw on creation was

 

   Disk 1

   Disk 2

      tv

   Disk 3

   Disk 4

   Disk 5

      tv

 

When I copied the show folders (one operation in  Krusader) that vast majority of the data went to disk 2 with only a few shows going to disk 5.

 

Why are Disk 1, 3 and 4 not participating in the share even though I specified All disks?

[trurl]

Share folders are only created on additional disks when they are needed, as determined by split level, allocation method, minimum free, and any include/exclude settings for the particular user share. There is also Global Share Settings, which can exclude specific disks from participating in user shares.

 

Another thing that can happen, and may be your situation. When copying a lot of data, some applications will create the empty folders in advance of creating the files to go in those folders. And those empty folders won't take up any significant space, so they may not trigger allocation method to cause them to be created on different disks. So, when it comes time to actually create the files, where they wind up may already be predetermined by where the folders were created.

 

See the Unbalance plugin if you want to get some thing moved to other disks:

 

https://forums.unraid.net/topic/43651-plug-in-unbalance/

 

[morallydubious]

Thanks for sticking with me but I am not sure this solves my problem. My understanding of Split level 2, Most Free, All disks is that the first 2 levels of the folder structure are split across All disks and the 3rd level is placed on a disk with the Most free space. For the example above that should result in each season folder and its contents on 1 disk but the next season could be on any of the other disks. For this to happen surely when I create the share in the first place each disk should have a folder for that share? For me only 2 disks had the tv folder. Even if the folder is not create on every disk I would expect it to be when the copy of all tv shows starts or when the next disk meets the 'Most Free' criteria.

 

In my case this did not happen. Only the original 2 disks actually received any data and the distribution was nowhere near even. As can be seen by the stats and the disk locations of the share.

 

 

[trurl]

26 minutes ago, morallydubious said:

the first 2 levels of the folder structure are split across All disks and the 3rd level is placed on a disk with the Most free space.

I'm having some trouble understanding this wording and how it relates to split level. Split level and allocation method are independent, and split level has precedence.

 

The point of split level is to make files below a certain level stay together. So, for example, depending on your folder structure, TV shows might be spread across all disks, but a particular season of a particular TV show might be stored together on the same disk.

[morallydubious]

Sorry that was confusing. Spread is a much better word. My tv shows are not being spread across all my disks as specified. Only 2 disks have any data and that isn't evenly distributed.

[trurl]

Are your sure this isn't what happened?

6 hours ago, trurl said:

When copying a lot of data, some applications will create the empty folders in advance of creating the files to go in those folders. And those empty folders won't take up any significant space, so they may not trigger allocation method to cause them to be created on different disks. So, when it comes time to actually create the files, where they wind up may already be predetermined by where the folders were created.

 

[morallydubious]

as far as the distribution of data is concerned its possible.... i suppose. New to unraid so don't know how it deals with this. If it is the case it doesn't explain why only 2 disks are listed as locations for the share and why second disk gets any data at all.

[morallydubious]

Update: I am running some tests transferring enough data into a new share (settings as above) and started by copying tv shows one at a time until the current disk is no longer Most Free. One tv show (with its subfolders) was copied that took the current drive way beyond the next Most Free drive but copied all seasons to the same drive. I copied another tv show and this was written to the next Most Free drive (this didn't have the share folder before the last copy).

 

At the moment it does look like its the way I bulk copied the folder in Krusader. To acheive the results I want I am going to have copy at the level I want to remain on the same disk ie tv/tvshow/season. I am really not sure how to that if the 2nd level (tvshow) doesn't already exist on the target. Looks like my scripting skills are going to get a brush up this evening 🙂

[morallydubious]

Thank you @trurl

 

Scripting would have taken too long unfortunately so i used an unBalance dryrun to get the commands required. They obviously weren't right being disk to disk but some quick find an replace later and i have the rclone copy commands to run in a batch. I ran the copy of my tv shows overnight and thie result is attached. Perfect! Much quicker than krusader for this type of operation.