Jump to content
magiin83

Cache_dirs scan level depth "auto" not enough?

61 posts in this topic Last Reply

Recommended Posts

Spinup groups are enabled in Disk Settings.  You can assign or change group names to each disk by clicking on the individual disks.  I assume that you could also edit the disk.cfg file and restart the array as well, but I'm not sure.  I do know if spinup groups are not enabled and you edit the group names in the file it gets changed back upon array startup.  Change them through the GUI.

 

I expected spinup groups to not be functional without group names assigned to disks.  But now I'm trying to determine if I have done little more than enable a default spinup group (hence all drives spinning up for the Mover this morning).  But something definitely changed for the better from my Kodi client perspective, and maybe this is all related.

 

Mover will always spin up all drives, regardless of there being files going to only 1 drive or not.

I'll take a closer look into spinup groups, I've heard enough back and forth about them, the positive and negative, but assumed they were just a part of unraid almost no one uses anymore.

Share this post


Link to post

My experience with Mover is that it never used to spin up multiple disks.  I'm not sure why it would behave any different than any other rsync (which is all it really does) or write operation to the array.  I could be wrong, but I disagree that this is how it should function.

Share this post


Link to post

My experience with Mover is that it never used to spin up multiple disks.  I'm not sure why it would behave any different than any other rsync (which is all it really does) or write operation to the array.  I could be wrong, but I disagree that this is how it should function.

 

My experience, it's rare when I've seen it not spin up all drives upon mover initiating.  I can think of times when it hasn't, but you're right, it shouldn't technically be spinning all drives as it is just rsync and pointing specifically to certain drives.

 

I've removed the spin-up group names for now, will report back with anything else I see.

Share this post


Link to post

Update:

Spin-up groups enabled, all groups are blank named, across all disks.

TV episodes, no issue whatsoever, download as many, import into sonarr, kodi gets updated, no spin ups.

 

Movies however, there's clearly a problem (which again this may go back to the issue of memory, things being pushed out of memory cache)

As soon as the movie finishes unpacking and Radarr picks it up, that initial move where the file is filename.partial as it moves into /mnt/cache/Movies.... all drives spin up.  Across the board, the second that movie begins its initial copy as the symlink.  6gb file for movies, versus 500megs files for those tv episodes which causes no disruption or spinning up.

Share this post


Link to post

Looks to me as though my latest change - to enable spinup groups without specifying a group name in the disks - is a workaround to an underlying problem in unRAID.  I still see random disks spinning up occasionally during various actions (including the Mover), but the spinups are not happening serially as before.  So it greatly lessens the impact, but doesn't change the fact that drives are spinning up with no obvious need. 

 

Ripped a movie - wrote to cache disk with no other spinups.

Envoked the Mover - two disks spun up immediately and the Mover wrote to one

Ran dlandon's File Activity script - GUI hung for 20 seconds and I found that three drives had spun up with read activity (only one showed any write activity in the output)

Clicked back and forth among root folders in Windows File Explorer for a while and saw two more drives eventually spin up.

 

I'm left to wonder if a.) most people don't enable spin down on their drives, b.) they just never notice this happening in their system, or c.) my configuration is somehow different. 

 

Out of ideas and about to drop $75 on memory that I probably don't need. 

Share this post


Link to post

Wait for the next release of unRAID (6.3.2).  LT turned on a feature for OSX that was called vfs_fruit.  This is a feature in samba to support OSX access to samba shares.  It seems that it turned on the samba extended attributes feature that is causing additional file system reads and writes that you are probably seeing here.  It was added in the 6.3.0rc.

 

It should clear up the excessive spin ups you are seeing here.

Share this post


Link to post

Honestly, that doesn't sound right to me.  Wasn't version 6.3.0 released at the beginning of the month?  I know for sure I have been fighting this issue for much longer than a couple of weeks, and I never run beta versions.  Do you possibly have the version mixed up?  6.2.x or earlier would make more sense to what I've been seeing. 

 

Thanks for your help dlandon.

Share this post


Link to post

Honestly, that doesn't sound right to me.  Wasn't version 6.3.0 released at the beginning of the month?  I know for sure I have been fighting this issue for much longer than a couple of weeks, and I never run beta versions.  Do you possibly have the version mixed up?  6.2.x or earlier would make more sense to what I've been seeing. 

 

Thanks for your help dlandon.

 

Before 6.3.0 was released there were 6.3.0rc versions.  You may also have another issue that is not related to the vfs fruit issue if you have been seeing it before 6.3.0.

Share this post


Link to post

I've always had different names on my Drives so even if I turn on Enable Disc Groups each drive would have its own Group since from what I understand you have to name two drives with the same name so they belong in the same group.

 

I also use Ember, XMBC aka Kodi now. I'd try to do some experimenting for you on spin ups and whatnot, but I've been moving data around on my drives the past few days since I've allowed myself to move things around and not group them up like I used to do. Data here, Data there and its messy and my OCD is going crazy it. ;)

Share this post


Link to post

Some comments:

 

* I would not blank the spin group names.  Clearing them all just makes them equal, which may cause all of the drives to spin up if any drive needs to spin up.  Better would be to give each one a unique name, such as host1 for disk1, host8 for disk8, and host90 for the cache drive.  By the way, I don't think anyone mentioned the cache drive above, in connection with spinup groups, but it too has one, in case someone wants it spun up the same time as a share is used.  You want the spinup group for the cache drive to be unique too.

 

* I really don't think most users are ever seeing spin up issues, at all.  It just works.  I really don't know why you both are having issues.

 

* It's possible there are a few other users, with unexpected spinups, but I don't think most believe it's a problem, just a very minor annoyance, so they ignore it.

 

* More memory is probably not going to help.  So long as you have enough memory, a minimum amount, then you should not need to add more.  4GB should be more than enough if you aren't doing anything with dockers or VM's.

 

* I still recommend only including the specific shares you want cached (in the Cache Dirs settings), and leave 'Scan user shares' set to No.

 

* There's one other reason possible for spinups, even after you've displayed a folder before without spinning up the drive.  In *some* file managers, if you hover over a file, intentionally or not, the tool or OS may open the file for metadata, and display it in a popup box (stuff like minutes, codecs, resolution, author, etc).

Share this post


Link to post

Some comments:

 

* I would not blank the spin group names.  Clearing them all just makes them equal, which may cause all of the drives to spin up if any drive needs to spin up.  Better would be to give each one a unique name, such as host1 for disk1, host8 for disk8, and host90 for the cache drive.  By the way, I don't think anyone mentioned the cache drive above, in connection with spinup groups, but it too has one, in case someone wants it spun up the same time as a share is used.  You want the spinup group for the cache drive to be unique too.

 

* I really don't think most users are ever seeing spin up issues, at all.  It just works.  I really don't know why you both are having issues.

 

* It's possible there are a few other users, with unexpected spinups, but I don't think most believe it's a problem, just a very minor annoyance, so they ignore it.

 

* More memory is probably not going to help.  So long as you have enough memory, a minimum amount, then you should not need to add more.  4GB should be more than enough if you aren't doing anything with dockers or VM's.

 

* I still recommend only including the specific shares you want cached (in the Cache Dirs settings), and leave 'Scan user shares' set to No.

 

* There's one other reason possible for spinups, even after you've displayed a folder before without spinning up the drive.  In *some* file managers, if you hover over a file, intentionally or not, the tool or OS may open the file for metadata, and display it in a popup box (stuff like minutes, codecs, resolution, author, etc).

 

I've put the host names back into the spin up group sections, with each drive having its own name.

 

Out of all that has been brought up with this issue, my issue seems to occur with the following, and only the following:

- Movies (once the download finishes, and Radarr/CP/whatever sees said files and begins to import them to the array via cache drive symlink, as soon as the files finish moving, immediately drives spin up. All of them.  There isn't a notification for anything going anywhere, all Radarr is doing is just importing said movie into the list.

TV does not have this issue and it DOES have a notification sent out yet nothing spins up for it, but this could go back to Sonarr having specific folder updates that properly work for kodi, not full library scan that CP/Radarr can only do for now.

 

Movies Open Files Log:

unrar 30844

3 2 /downloads/inter/Sexy.Beast.2000.720p.BluRay.DD5.1.x264-EbP.#27 (working directory)

/downloads/inter/Sexy.Beast.2000.720p.BluRay.DD5.1.x264-EbP.#27/iDJGuQB7yMH7sBfGNkkEWt6Nc2oFMr7WAXX2 ...

/downloads/Movies/Sexy.Beast.2000.720p.BluRay.DD5.1.x264-EbP/_unpack/KRUZjENyfY7g8XMoQDnGGhRftOY1/1P ...

 

shfs 10638

1 1 /mnt/cache/Movies/Sexy.Beast.(2000)/Sexy.Beast.2000.Bluray-720p.x264.AC3.mkv.partial~

 

Compared to

TV Open Files Log:

shfs 10638

1 1 /mnt/cache/Television/Black.Sails/Season.03/S03E01.-.XIX.DVD.x264.AAC.mkv.partial~

 

mono

/downloads/TV/Black.Sails.S03E01.BDRip.X264-DEFLATE-Obfuscated/1GaxrlCWwuFjXL1AXBjT0bpEK.mkv.backup~

(/media refers to docker host path /mnt/user/Television) /media/Black.Sails/Season.03/S03E01.-.XIX.DVD.x264.AAC.mkv.partial~

 

- Browsing a share via windows explorer (granted, this could be anything from the folder view structure to just accidentally hovering over the directory too long)

 

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.