Share not utilizing cache or additional drives in the array

[Old James]

I have six 10TB drives in my array, plus cache. I set up my /data share to use the cache drive and all of the disks, but it appears to have never touched the cache drive (as of two days ago there was no /data on the cache, I manually created it in hopes of fixing the issue) and will only write to /disk1

 

The share is set up with the following settings:

Use cache pool: Yes

Select cache pool: cache

Enable Copy-on-write: Auto

Allocation method: High-water

Minimum free space: 1TB

Split level: Automatically split only the top two directory levels

Included disk(s): All

Excluded disk(s): None

Share status: Share contains data

 

I have this share mapped into two docker containers as /data <-> /mnt/user/data which have been loading data into the share, but are now reporting there share is full and have stopped functioning. /disk1 is completely full, but the remaining 5 disks have about 30TB free.

 

I previously used mc to move some files to /disk3/data (when /disk1 first started reaching capacity) and my dockers can see the moved files, so reading from the share perfectly fine the problem seems to be just writing.

 

Do I have something configured incorrectly that is causing this, or should I report it as a bug?

 

EDIT: To add more context, if it helps. I used to have separate /media and /download shares. I created the /data share a few weeks ago so I could make /media and /download sub-directories of the same share and allow the docker apps (with reconfigured mapping) do hard-linking when moving files between the directories instead of rewriting all the 1s and 0s of a move between shares.

I then used mc to move the files from the old /media share into the new /data/media.

 

EDIT 2: For anyone future person with this issue, see my last post here with what I believe is the cause and solution. Will add another edit in a couple of days to confirm.

 

Edited December 9, 2022 by Old James

[JorgeB]

What size is you cache device/pool? Also note that split level overrides allocation method.

[trurl]

6 minutes ago, Old James said:

mnt/usr/data

Make sure this typo is not the actual host path you specified. 

 

The user shares are in mnt/user

[trurl]

Are you sure the files were never on cache? Maybe they were already moved to the array. 

 

Attach Diagnostics to your NEXT post in this thread. 

[Old James]

2 minutes ago, JorgeB said:

What size is you cache device/pool? Also note that split level overrides allocation method.

 

I actually have 2 cache drives: a 250 GB for appdata and domains, and a 1TB for everything else. This share uses the 1TB which has 900GB free.

 

  

7 minutes ago, trurl said:

Make sure this typo is not the actual host path you specified. 

 

The user shares are in mnt/user

 

You are correct, that was a typo. I am going back to fix.

 

  

6 minutes ago, trurl said:

Are you sure the files were never on cache? Maybe they were already moved to the array. 

 

Attach Diagnostics to your NEXT post in this thread. 

 

A few days ago as the drive first exceeded 9TB utilization (I thought based on my settings it wouldn't allow it to cross this line) I looked at the cache directory and there was no /data on that drive (would UNRaid delete the share folder when running mover?).

Diagnostics attached to this post. Thanks.

tower-diagnostics-20221209-1024.zip

[trurl]

You seem to have 3 pools but only 2 drives. I think maybe something got messed up when you renamed your pools.

[JorgeB]

10 minutes ago, Old James said:

and a 1TB for everything else.

 

41 minutes ago, Old James said:

Minimum free space: 1TB

With this setting any writes will skip cache, since there will never be 1TB free.

[Old James]

1 minute ago, JorgeB said:

 

With this setting any writes will skip cache, since there will never be 1TB free.

Ah, I was thinking of that applying to the HDDs but not the cache itself. That makes a ton of sense!

 

One problem solved.

[Old James]

7 minutes ago, trurl said:

You seem to have 3 pools but only 2 drives. I think maybe something got messed up when you renamed your pools.

I had an NVME drive assigned as cache*, but removed it to pass through bare metal to a Windows VM.

 

When the VM is shutdown, that drive currently appears as an unassigned device instead of a cache pool.

EDIT: The NVME spent less than a day as a cache drive and I don't think I ever assigned a share to use it.

Edited December 9, 2022 by Old James

[Old James]

Ok, I think my problems are the following:

 

1. My minimum space free was too high, preventing use of the cache drive
2. My split level was too low (2) and needs to be higher (4) to split the correct sub-directories to other drives.

 

I will give that a try and edit the first post in a couple of days it this fixed it.

[JorgeB]

5 minutes ago, Old James said:

My split level was too low (2) and needs to be higher (4) to split the correct sub-directories to other drives.

That would be the other way around.

[Old James]

Just now, JorgeB said:

That would be the other way around.

Oh, I have /data/media/movies/family_videos_2022/
I want it to split when I create /data/media/movies/family_videos_2023/
I think level 4 is what I want for this structure, correct?

 

When you say the other way around is that my split level should remain 2 vs 4, or calling 2 low and 4 high is the wrong terms?

[itimpi]

3 minutes ago, Old James said:

Oh, I have /data/media/movies/family_videos_2022/
I want it to split when I create /data/media/movies/family_videos_2023/
I think level 4 is what I want for this structure, correct?

 

When you say the other way around is that my split level should remain 2 vs 4, or calling 2 low and 4 high is the wrong terms?

That looks like 4 is the correct value if you want family_videos_2023 to exist on multiple drives.

[JorgeB]

20 minutes ago, Old James said:

When you say the other way around is that my split level should remain 2 vs 4, or calling 2 low and 4 high is the wrong terms?

You are correct, I got confused.

[itimpi]

47 minutes ago, JorgeB said:

You are correct, I got confused.

That is my normal state