Split Level and Allocation Method

[PlayLoud]

Greetings all,

 

Unraid 6.9.2

 

Array

1x 10TB Parity

4x 10TB Data

 

So, I recently decided to reorganize my media. Previously, I had all my movies and tv shows in a share called "media".

 

As I read about split-level, I decided I wanted all files for any given movie/show to be on the same drive. I figured the best way to do this was to copy off all my media, create new shares (Movies and TV Shows instead of Media), set the Split Level, and copy it back.

 

My folder structure is as follows

 

Movies (share)

   Movie name (folder)

      Movie.mkv

 

TV Shows (share)

   Show name (folder)

      Season 1 (folder)

         1x01.mkv

         1x02.mkv
      Season 2 (folder)

         2x01.mkv

         2x02.mkv

etc

 

My Allocation Method is set to High-water.

 

After reading about Split-Level, I thought I understood it and set it for "Automatically split only the top level directory as required."

I thought this was put all files of any particular movie or TV show folder (including all seasons) on the same drive.

 

After creating the new shares as described above, I used Krusader to start copying my media back to the array.

My new Movies share is set to only use Drive 3. Since I currently have only 4.17TB of movies, this transferred as expected.

 

TV Shows are set for Disk 1 and Disk 2.

However, I wake up this morning to find that all ~7TB of TV shows are on Disk 1. I was expecting ~5TB of shows on disk 1 due to the High-water allocation. I knew it might go higher as if a series had started on disk 1, the split level would force it to stay on that disk. However, I wasn't expecting the entire share to be on Disk 1.

 

I figured I misunderstood the Split Level, and change it to "Automatically split only the top two directory levels as required", delete everything off of it and start the transfer again. The same thing happening (currently 5.79TB on Disk 1). The only thing that is currently on Disk 2 are 2x series.xml files from shows that are on Disk 1. Now I'm thinking the first Split-Level option was correct in that regard.

 

But now I am still confused. Shouldn't the High-water allocation have caused a split at ~5TB (once the current show was finished).

 

I know I could always unbalance some shows to balance the drive a little better, but I'd really like to know what I am not understanding.

 

Thanks in advance for any insight.

skynet-diagnostics-20220310-2228.zip

Edited March 11, 2022 by PlayLoud
Attachment

[itimpi]

Splitting only the top level is what you want for the distribution you say you want.

 

I believe that the problem with your TV shows is that Krusader creates all the folders before it starts copying files into them.   Since cresting a folder takes almost no space they all get created on the first drive and then the Splut Level setting constrains the files to that drive.    If you copy the files back in smaller increments then you should get the files split across the drives.   You would also get it if you used a copy method that does not start by creating all the folders required. 

[PlayLoud]

7 hours ago, itimpi said:

Splitting only the top level is what you want for the distribution you say you want.

 

Thank you. I guess my original understanding was correct.

 

 

7 hours ago, itimpi said:

Since cresting a folder takes almost no space they all get created on the first drive and then the Split Level setting constrains the files to that drive.  If you copy the files back in smaller increments then you should get the files split across the drives.

 

Ah, I can see how that would mess it up.

 

 

7 hours ago, itimpi said:

If you copy the files back in smaller increments then you should get the files split across the drives.

 

That seems easy enough. It would only be something I have to do on the initial transfer. Adding folders afterwards would be much smaller chunks where that would not be necessary.

 

 

7 hours ago, itimpi said:

You would also get it if you used a copy method that does not start by creating all the folders required. 

 

Do you know a copy method I could use for this? The data is currently sitting on an unassigned device on the same system.

[itimpi]

11 minutes ago, PlayLoud said:

Do you know a copy method I could use for this? The data is currently sitting on an unassigned device on the same system.

I think you would be OK if you used ‘cp’ or ‘mc’ (midnight commander) from a console session.  Not sure about rsync.    Another possibility is doing it over the network but then network speed becomes a constraint.

[PlayLoud]

11 minutes ago, itimpi said:

I think you would be OK if you used ‘cp’ or ‘mc’ (midnight commander) from a console session.  Not sure about rsync.

 

Thank you for all the info.

 

I changed the split level back and deleted the folders that brought the total over 5TB (this also deleted those small .xml files that were on Disk 2).   Then I started transferring again, and the remaining folders started going to Disk 2.   It looks like all will be good once this last couple TB transfers, but I will keep your suggestions in mind if this ever comes up again.

 

 

16 minutes ago, itimpi said:

Another possibility is doing it over the network but then network speed becomes a constraint.

 

That's one area where I have no bottleneck. 10g