Determine which disks a directory resides on?

[mrow]

Is there any easy way to determine which disks a specified directory resides on?

 

For instance, is there an easy way to see that the directory "The Sopranos" in my TV Shows user share resides on disks 1, 4, and 7 without having to manually browse each disk?

[Wody]

In Unraid 5, you can browse the share in the webGUI, by clicking the magnifying-glass-on-folder-icon, and it will tell the disk(s) the content is on on the right, as you browse through the share (and its subfolders by clicking those)

[Joe L.]

Is there any easy way to determine which disks a specified directory resides on?

 

For instance, is there an easy way to see that the directory "The Sopranos" in my TV Shows user share resides on disks 1, 4, and 7 without having to manually browse each disk?

yes, but the methods available depend on the version of unRAID you are running.

 

On any version you can type:

find /mnt/disk* -type d -iname "The Sopranos" -print

[mrow]

In Unraid 5, you can browse the share in the webGUI, by clicking the magnifying-glass-on-folder-icon, and it will tell the disk(s) the content is on on the right, as you browse through the share (and its subfolders by clicking those)

 

 

I'm using 5.0 so this works for me. Thanks for the tip!

[mrow]

Is there any easy way to determine which disks a specified directory resides on?

 

For instance, is there an easy way to see that the directory "The Sopranos" in my TV Shows user share resides on disks 1, 4, and 7 without having to manually browse each disk?

yes, but the methods available depend on the version of unRAID you are running.

 

On any version you can type:

find /mnt/disk* -type d -iname "The Sopranos" -print

 

 

Joe: is there a way to pipe the results of your find command to move everything to one specified disk? I must have turned turned off split levels off for my TV Shows user share at some point, for some reason, and now shows are split up all over the place. I've got 150 TV series in that share and some shows are split up across 9 disks. Doing it all manually would take 5 years.

[JonathanM]

Joe: is there a way to pipe the results of your find command to move everything to one specified disk?

Maybe, but why not just move the affected shows to the cache drive, filling it as much as you can, then fix your allocation method and let the mover sort it out? If your TV directory is something like /mnt/user/TVShows/A-team/Season01/Episode1.avi, then you would just create a TVShows folder on the cache drive and move A-team to /mnt/cache/TVShows/A-team, and manually start the mover. If your cache drive is large enough, you wouldn't have to do it more than a few times.

[garycase]

I've got 150 TV series in that share and some shows are split up across 9 disks.

 

So ??  The whole idea of a share is that it doesn't matter ... UnRAID will manage it as a single disk when you access it.  If they're indeed split in a manner that causes excessive spinups then you may want to do a bit of restructuring;  but otherwise it really doesn't matter.

 

 

Doing it all manually would take 5 years.

 

Can you spell "exaggerate" ??    8)  (as I'm sure you know)

 

The simplest (not necessarily the fastest) approach is to just select the entire share you want to consolidate;  copy it to another location [not on UnRAID ... perhaps a large external drive, or another UnRAID server];  then delete the entire contents of the share;  and then copy it back, after adjusting the included disk(s) and split level for the share to what you want.    This basically takes about 5 MINUTES of "your time" .. although obviously many hours of computer-time for the actual data movement.    But certainly a lot less than 5 years 

[mrow]

If they're indeed split in a manner that causes excessive spinups then you may want to do a bit of restructuring;  but otherwise it really doesn't matter.

 

That's exactly why I want to clean it up. I've got instances where literally the only thing for a TV series that is on a disk is the episode thumbnails for one season of the show. So if those thumbnails need to be read, that whole disk will spin up just for a couple 30KB files. Granted this isn't very often but it's just one of those things that bothers me in the back of my head enough to make me want to fix it.

 

The simplest (not necessarily the fastest) approach is to just select the entire share you want to consolidate;  copy it to another location [not on UnRAID ... perhaps a large external drive, or another UnRAID server];  then delete the entire contents of the share;  and then copy it back, after adjusting the included disk(s) and split level for the share to what you want.    This basically takes about 5 MINUTES of "your time" .. although obviously many hours of computer-time for the actual data movement.    But certainly a lot less than 5 years 

 

Not really feasible with the equipment I currently have. My TV Shows share is 6.24TB. I guess jonathanm's suggestion of moving stuff to the cache drive in sections is the best option.

[JonathanM]

My TV Shows share is 6.24TB. I guess jonathanm's suggestion of moving stuff to the cache drive in sections is the best option.

If you really want to get OCD, figure out the data split necessary to fit all the shows in alphabetical order on consecutive disks. My first TV show disk has 24 through Doctor Who, the second has E-Ring through Law and Order, the third has Leverage through WKRP. Documentaries and stuff have their own disk, and reality shows plus anything else I watch and delete is on a separate disk. I like to keep active disks less than 3/4 used, collection disks I tend to fill up all the way.

 

If I were you, I'd figure out which disk would be the first in the TV share, and exclude it from the share temporarily, so nothing gets moved there. Then I'd move everything I could off of that disk share, and let the mover find spots for it wherever it can. Then include only that disk in the user share, select exactly which shows you want to be there out of the user share, and put them on the cache drive and run the mover again. Lather, rinse, repeat.

 

If you aren't quite that OCD, then just grabbing chunks of shows out of the user share and dropping them on the cache drive should gather them all back up as long as your split level and allocations are sane.

[garycase]

Not really feasible with the equipment I currently have. My TV Shows share is 6.24TB. I guess jonathanm's suggestion of moving stuff to the cache drive in sections is the best option.

 

Doing it in sections would work okay.    Or you could simply do it by series  => basically the same thing I suggested, but in pieces.    First, change the includes and split level as you want it to be.  Since this only impacts writes, it won't hurt anything that's already there.

 

Now copy as much as your secondary storage location can hold ... perhaps 500GB, 1TB, etc.    Doesn't have to be the entire 6.24TB.    But don't copy arbitrarily ... copy one (or more) series to the location; then delete them from UnRAID; and then simply copy them back (the copy back will "obey" the split level and locations limits).    Then delete them from your temporary location (freeing up space for the next batch) and proceed.    Once you've done that for all your series, you've got it like you want it !!

 

[mrow]

I've got 7TB free split pretty evenly across all my disks so I'm going to use a split level of 2 as there shouldn't be a problem fitting a whole series on a single disk. I'm moving a few series at a time into a folder called .temp on the cache drive, then deleting those series from the user share directory so the series is removed from all array disks and then moving them from .temp to the TV Shows user share. Still a bit of a pain but not too bad. I have a 4TB preclearing right now I could swap in as the cache disk and move almost 2/3s of the shows at once so we've see. Not sure I want to do that on the fluke chance the 4TB suddenly dies and I'm up shit creek.

[mrow]

So far I've consolidated 6 series. One strange occurrence so far…how strictly enforced are split levels supposed to be? With one of the series, for some reason it it put everything on disk 4 except for five 50KB jpegs which are the box art for each season and are stored in the root directory of the series, which it dropped on disk 1. What's more strange is that disk 4 has about 1TB more free space than disk 1. The TV Shows user share is set to split level 2 so anything for a series should go in the same directory.

 

Any ideas?

[garycase]

So far I've consolidated 6 series. One strange occurrence so far…how strictly enforced are split levels supposed to be? With one of the series, for some reason it it put everything on disk 4 except for five 50KB jpegs which are the box art for each season and are stored in the root directory of the series, which it dropped on disk 1. What's more strange is that disk 4 has about 1TB more free space than disk 1. The TV Shows user share is set to split level 2 so anything for a series should go in the same directory.

 

Any ideas?

 

What's your directory structure?    Are you SURE that the series name is within the appropriate split level?    Also, what's the allocation method set to?

 

[mrow]

So far I've consolidated 6 series. One strange occurrence so far…how strictly enforced are split levels supposed to be? With one of the series, for some reason it it put everything on disk 4 except for five 50KB jpegs which are the box art for each season and are stored in the root directory of the series, which it dropped on disk 1. What's more strange is that disk 4 has about 1TB more free space than disk 1. The TV Shows user share is set to split level 2 so anything for a series should go in the same directory.

 

Any ideas?

 

What's your directory structure?    Are you SURE that the series name is within the appropriate split level?    Also, what's the allocation method set to?

 

High water, and disk 4 had way more space left than disk 1 (or any other disk for that matter) so that shouldn't have mattered. Split levels were definitely correct. Even if I had split level set to 3, those files still should have gone on the same disk because they were in the root directory of that series. Only sub-directories of the series directory should go to other disks at split level 3. But I have the TV Shows split level set to 2 so there should be no way any files or sub-directories of the main series directory should be on any other disk. At split level 2, any files or sub-directories of the "Seinfeld" directory in the TV Shows user share, for instance, should all be on the same disk.

 

 

EDIT: So here is a screen shot of the directory structure. As you can see, the all series are sub directories of the TV Shows user share, and at split level two all files and sub-directories should reside on the same disk as the root directory of the series.

 

 

 

 

 

 

EDIT 2: Here it is in the browser too so you can see the exact path.

 

 

[garycase]

So if I'm reading this right, you have TV Shows - American Horror Story - Season 1 - ... etc.

 

Are the jpegs that were on a different disk the ones in the "American Horror Story" folder?  ... or were they under Season 1, or Season 2, etc. ??    What happens makes sense if the latter; but not the former.

 

As for Disk 4 having 1TB more than any other drive -- whether that matters depends (as I assume you know) on the relative size of the drives.

 

It does seem, however, that your jpegs should have been on the same disk as the rest of the show!!

 

[mrow]

So if I'm reading this right, you have TV Shows - American Horror Story - Season 1 - ... etc.

 

Are the jpegs that were on a different disk the ones in the "American Horror Story" folder?  ... or were they under Season 1, or Season 2, etc. ??    What happens makes sense if the latter; but not the former.

 

As for Disk 4 having 1TB more than any other drive -- whether that matters depends (as I assume you know) on the relative size of the drives.

 

It does seem, however, that your jpegs should have been on the same disk as the rest of the show!!

 

 

They were in the American Horror Story directory. Even a split level 2 they should still all be on the same disk though.

[garycase]

They were in the American Horror Story directory. Even a split level 2 they should still all be on the same disk though.

 

Agree.  Not sure why they would have been stored elsewhere UNLESS there was already a folder with that name on the other disk.    Then it probably "confuses" the split-level logic.

 

One simple way to be sure everything goes where you want:  Just copy to the disk shares instead of the user share.    And be sure you don't have any level 2 folders that exist on more than one disk.    Together, that will get you set up so everything in the future that you copy to the user shares will be structured exactly as you want it.

 

[mrow]

They were in the American Horror Story directory. Even a split level 2 they should still all be on the same disk though.

 

Agree.  Not sure why they would have been stored elsewhere UNLESS there was already a folder with that name on the other disk.    Then it probably "confuses" the split-level logic.

 

One simple way to be sure everything goes where you want:  Just copy to the disk shares instead of the user share.    And be sure you don't have any level 2 folders that exist on more than one disk.    Together, that will get you set up so everything in the future that you copy to the user shares will be structured exactly as you want it.

 

 

Once I copy the series to my .temp directory on my cache drive I issue an "rm -rf /mnt/user/"TV Shows"/<TVseries>". I then verify that the series directory is gone from the user share. So there is either a bug where a directory still exists in user share directory on an individual disk and doesn't show up in the when browsing that user share, or there's a bug with the split level system.

[garycase]

Once I copy the series to my .temp directory on my cache drive I issue an "rm -rf /mnt/user/"TV Shows"/<TVseries>". I then verify that the series directory is gone from the user share. So there is either a bug where a directory still exists in user share directory on an individual disk and doesn't show up in the when browsing that user share, or there's a bug with the split level system.

 

Have you changed the "Includes" for the share?  If so, the rm command may not be deleting the folder from drives that are no longer part of the share.

 

[dgaschk]

I've got 7TB free split pretty evenly across all my disks so I'm going to use a split level of 2 as there shouldn't be a problem fitting a whole series on a single disk. I'm moving a few series at a time into a folder called .temp on the cache drive, then deleting those series from the user share directory so the series is removed from all array disks and then moving them from .temp to the TV Shows user share. Still a bit of a pain but not too bad. I have a 4TB preclearing right now I could swap in as the cache disk and move almost 2/3s of the shows at once so we've see. Not sure I want to do that on the fluke chance the 4TB suddenly dies and I'm up shit creek.

 

The split level is wrong. Split level refers to the allowable split level.

 

1 - Allow "TV Shows" to split

2 - Allow "TV Shows" and "American Horror Story" to split

3 - Allow "TV Shows", "American Horror Story", and "Season 1" to split

4 -  etc...

 

Split level 2 or greater allows the files in "American Horror Story" to split across disks. Only the values of 0 or 1 will keep the contents of "American Horror Story" together.

 

See here: http://lime-technology.com/wiki/index.php/Un-Official_UnRAID_Manual#Split_level

[mrow]

I've got 7TB free split pretty evenly across all my disks so I'm going to use a split level of 2 as there shouldn't be a problem fitting a whole series on a single disk. I'm moving a few series at a time into a folder called .temp on the cache drive, then deleting those series from the user share directory so the series is removed from all array disks and then moving them from .temp to the TV Shows user share. Still a bit of a pain but not too bad. I have a 4TB preclearing right now I could swap in as the cache disk and move almost 2/3s of the shows at once so we've see. Not sure I want to do that on the fluke chance the 4TB suddenly dies and I'm up shit creek.

 

The split level is wrong. Split level refers to the allowable split level.

 

1 - Allow "TV Shows" to split

2 - Allow "TV Shows" and "American Horror Story" to split

3 - Allow "TV Shows", "American Horror Story", and "Season 1" to split

4 -  etc...

 

Split level 2 or greater allows the files in "American Horror Story" to split across disks. Only the values of 0 or 1 will keep the contents of "American Horror Story" together.

 

See here: http://lime-technology.com/wiki/index.php/Un-Official_UnRAID_Manual#Split_level

 

 

Well that would account of one of the season directories was on another disk, but from the description on the wiki, split levels are by directories and not files, so shouldn't files in the root directory of American Horror Story not stay in that same directory? That's what this makes it sound like:

 

For Movies use a split level = 2. This allows the "SD Movies", "HD Movies" and "Kids Movies" folders to be placed on every disk and it keeps each individual movie folder on a single disk. This way, any single movie folder and the contents of the movie folder will remain on a single disk.

For TV_Shows use a split level of either 1 or 2. A split level of 1 will keep each TV series on a single disk and split level of 2 will keep each season on a single disk. The split level of 2 means that the complete TV series can be stored on multiple disks, however each individual season of that TV series will be on a single disk.

 

The issue I came across was files in the root of American Horror Story ended up on another disk, not that a season ended up on another disk. Or am I still reading that incorrectly?

 

 

EDIT: See bolded lines from the wiki:

 

Level 2

The top level Media share can be created on every disk.

The SD Movies, HD Movies, Kids Movies and TV Shows folders can be created on every disk.

Each Movie Folder and TV Show Folder must remain on a single disk.

This setting may work well. It will keep each movie and each TV series together on a single disk.

This setting may give issues because it keeps each TV series on a single disk. So, a disk may fill as new TV seasons are added to a TV show which is on a disk which is close to full.

[itimpi]

That wiki article is saying that the third level can be placed on any disk (this includes files and folders).  However if the third level is a folder then any contents of that folder are now constrained to a single disk.

[mrow]

That wiki article is saying that the third level can be placed on any disk (this includes files and folders).  However if the third level is a folder then any contents of that folder are now constrained to a single disk.

 

I understand that, however, the files that ended up on another disk were on the second level, not the third. See my screen shots. The files that ended up on another disk were those jpegs in the root of American Horror Story, so they were in the second level.

[dgaschk]

The system is working correctly. What problem are you trying to solve?

 

Again, "Split level 2 or greater allows the files in "American Horror Story" to split across disks. Only the values of 0 or 1 will keep the contents of "American Horror Story" together."

 

Are you choosing a value greater than 1? If so, the contents of  "American Horror Story" may be spread over several disk.

[mrow]

I was under the impression that split level only death with folder splitting, not files too. I thought a split level of 2 should have kept all files, not folders, in the top level of American Horror Story.