[SOLVED] Share is bigger than it should be (3x8TB HDDs - share shows 27TB free)

[WhiteTowerBlackNode]

Hi,

 

 

I have decided to add three new 8TB IronWolf HDDs and create a new share that only uses those three new HDDs, thus I should have a 24TB share.

I also excluded the three new disks from every other share.

 

The new share shall be my new Plex share (named "Plex"), but as I already have an older Plex share (named "Entertainment") I still have to do some sorting before copying everything over. I started with stuff I haven't seen yet, which is "just" 2TB of data.

 

But the Unraid web GUI shows the share as having 27.2 TB free space and Finder says, it still has 27.18 TB of free space. I'd like to know, where those extra 5TB are coming from (27TB "free" - 24TB actual size + 2 TB of already used data on it).

 

I used binhex krusader to COPY the data from the old share to the new share, as I do not understand the MOVE command (a test of moving a 100GB folder from the old share to the new share did result in the 100GB folder almost instantly being visible on the new share without Krusader having any kind of window showing the move process, and the HDDs did not show any big reading and writing activities) yet.

 

Does anyone have any idea? Do you need any other information or logs I may have forgotten?

 

 

Thanks.

 

 

PS: the new user share also does show using two other 8TB drives when using the web GUI (maybe due to the 100GB folder I wanted to move residing on those drives, but I moved that back and then copied it) and its file browsing feature.

But it only shows those two drives in one of the folders on the root, once I go deeper, it only shows one of the new drives as storage unit. Strange.

 

[JorgeB]

Do you have cache? it can be part of the share. If not post a main page screenshot as well as the diagnostics.

[itimpi]

This could be due to the fact that the share actually has some of its files on another drive.   Setting the share properties to only use some of your drives does not make unRAID automatically move any files that logically belong in this Share but are actually physically on other drives.   In this scenario you will still see the files that are on other drives - it is just new files that are restricted to the specified drives.

[bonienl]

Go to "Shares" and click on compute of the share in question. It will show on which disks the share is present.

[WhiteTowerBlackNode]

22 minutes ago, johnnie.black said:

Do you have cache? it can be part of the share. If not post a main page screenshot as well as the diagnostics.

Cache has been disabled, as I will move TBs of data and the cache is only a 500GB SSD.

 

 

Here are the diagnostics:  whitetower-diagnostics-20180910-1740.zip

 

21 minutes ago, itimpi said:

This could be due to the fact that the share actually has some of its files on another drive.   Setting the share properties to only use some of your drives does not make unRAID automatically move any files that logically belong in this Share but are actually physically on other drives.   In this scenario you will still see the files that are on other drives - it is just new files that are restricted to the specified drives.

 

The share "Plex" is new, it has been created after the three new 8TB disks have been added. During the creation of the new share, it was limited to those three new disks.

Thanks for the explanation of the moving. It is probably much faster that way, if one does not do something like I do (limit one share to only three disks). I can live with that. I guess the copy process is the one to go then, as the share will see that as "new" files.

 

 

18 minutes ago, bonienl said:

Go to "Shares" and click on compute of the share in question. It will show on which disks the share is present.

 

Ah, thanks for that click trick. 

It looks like this:

 

 

 

And thank you for your fast answers, strangely I cannot get in front of the first quote. Hmm.

[JorgeB]

Problem is that the share exists on disks 6 and 7, so they are used for the total space, delete that folder on those disks.

[bonienl]

Just now, johnnie.black said:

Problem is that the share exists on disks 6 and 7, so they are used for the total space, delete that folder on those disks.

Exactly that is what the mesaage "Share is outside the list of designated disks" indicates. The share is present on disks which are not defined in your include or exclude rules

[WhiteTowerBlackNode]

7 minutes ago, johnnie.black said:

Problem is that the share exists on disks 6 and 7, so they are used for the total space, delete that folder on those disks.

 

That is probably due to the MOVE I tried with Krusader. I am currently copying the specific folder to my desktop and see what happens when I delete it on the share. Oh, the copy process has already finished. Wired LAN is so much faster than that other W-LAN.

Am doing a restart of unRAID now.

 

Nope, did not work. 

 

 

5 minutes ago, bonienl said:

Exactly that is what the mesaage "Share is outside the list of designated disks" indicates. The share is present on disks which are not defined in your include or exclude rules

I will check again, but I did exclude it according to my memory. It was just yesterday.

 

 

 

 

[bonienl]

8 minutes ago, WhiteTowerBlackNode said:

I will check again, but I did exclude it according to my memory. It was just yesterday.

Those shares already existed, they will not be (re)moved when changing the exclude rules. You need to do that manually.

 

Btw it is better to define a single rule only, e.g "Include disk 8,9,10" implies an exclude of all other disks and doesn't need to be defined.

[trurl]

When you tried to move the files, linux simply moved them on the same disk they were already on. Since all the user shares are at /mnt/user, it considers them as the same location and so instead of copying from source to destination and then deleting from source, like it would do with a move between disks, it just "renamed" their path so they stayed on the same disk.

[trurl]

Since you are now in a position where you are needing to actually force it to move from one disk to another, it is very important that you don't mix disks and user shares when moving or copying files. Only specify a disk for the source and a disk for the destination. See here if you want to get into the details:

 

https://forums.unraid.net/topic/32836-user-share-copy-bug/

 

 

[WhiteTowerBlackNode]

28 minutes ago, bonienl said:

Those shares already existed, they will not be (re)moved when changing the exclude rules. You need to do that manually.

 

Btw it is better to define a single rule only, e.g "Include disk 8,9,10" implies an exclude of all other disks and doesn't need to be defined.

 

Which shares already existed?

 

The "Plex" share has been created yesterday, the "Entertainment" share has existed for a year.

The Plex share does only include disk 8,9,10 and exclude everything else (but if a single rule is better, I will change accordingly). The Entertainment share did only include disks 1 to 7 and I did exclude disks 8 to 10.

 

I copied from share Entertainment to share Plex, thus unRAID should have not included disks 6 and 7 somehow, even with only 0 bytes. I am just confused where the free space comes from. (It probably comes from the free space of disks 6 and 7.)

 

26 minutes ago, trurl said:

When you tried to move the files, linux simply moved them on the same disk they were already on. Since all the user shares are at /mnt/user, it considers them as the same location and so instead of copying from source to destination and then deleting from source, like it would do with a move between disks, it just "renamed" their path so they stayed on the same disk.

Thanks for the explanation. I suppose it is as moving a file from folder 1 on volume A to folder 237 on volume A, as it is the same physical disk and unRAID sees the ARRAY as physical disk and shares as "root folders".

 

23 minutes ago, trurl said:

Since you are now in a position where you are needing to actually force it to move from one disk to another, it is very important that you don't mix disks and user shares when moving or copying files. Only specify a disk for the source and a disk for the destination. See here if you want to get into the details:

 

https://forums.unraid.net/topic/32836-user-share-copy-bug/

 

 

That is why I limited the old share Entertainment to disks 1 to 7 and the new share Plex to disks 8 to 10.

 

I will see how it develops and may start again (which is no problem for that).

[trurl]

1 hour ago, WhiteTowerBlackNode said:

That is why I limited the old share Entertainment to disks 1 to 7 and the new share Plex to disks 8 to 10.

But as you have seen, those disk restrictions won't necessarily make things wind up on the disks you intend if you are just referencing the user shares when moving files.

 

Don't know how you have Krusader setup. Is it able to access the disks directly the way you have it configured, or just the user shares?

[WhiteTowerBlackNode]

7 hours ago, trurl said:

But as you have seen, those disk restrictions won't necessarily make things wind up on the disks you intend if you are just referencing the user shares when moving files.

 

Don't know how you have Krusader setup. Is it able to access the disks directly the way you have it configured, or just the user shares?

Krusader is able to do both, it can access the shares and the individual disks (just saw that disk 5 had a folder for a documentary, but no content, probably because the content is on another disk - quite groovy).

 

I suppose using MOVE with Krusader was my error, as it did not care about the shares I setup. Bastard. Or I am simply not well read when it comes to that, as I can see with Ubuntu 18.04 LTS Beaver Shite, which I cannot simply VNC into due to GNOME. Well, I suppose I look for another Linux distro.

[trurl]

I don't bother with Krusader or any other docker to do file management on my server. unRAID has mc (Midnight Commander) builtin. Plenty of information about Midnight Commander on the internet.

[WhiteTowerBlackNode]

On 9/11/2018 at 3:57 AM, trurl said:

I don't bother with Krusader or any other docker to do file management on my server. unRAID has mc (Midnight Commander) builtin. Plenty of information about Midnight Commander on the internet.

 

I use Krusader because SpaceInvader (YouTube) did recommend it when synching directories on two different servers (I have a smaller unRAID server with older disks for backing up the bigger server (not completely, only the stuff, that isn't re-creatable)), thus my use of that docker. I know about MC, but didn't delve too deep into it about its capabilities.

 

 

 

 

AS FOR THE ISSUE at hand, it got solved with deleting the share and doing a new one, which got me to slowly turn all the AVC based video content (entertainment such as cat videos of course) into HEVC based content and only populate the newer share with content using HEVC. Saves a lot of storage and I don't mind.