Not enough free space on drive (I have 5.48TB Free)

[trurl]

You have a cache-no share, but it has files on cache. Possibly this is a result of what you have been trying to do.

 

And your system share has files on the array. This might also be related to your full cache since cache-prefer will overflow, or it might have already had files on the array before this started, since typically that share will not have any new files.

 

You might want to stop what you have been doing and run mover. Then going forward use the copy to destination then delete from source workaround I mentioned in that link.

[trurl]

Also a lot of errors in syslog related to your full cache. Hopefully it won't have corruption.

[burn0u7]

47 minutes ago, SopraNo3 said:

Hello,

 

I have the same issues. I do not understand how this works. I got a lot space left (75TB+) so most probably it has something to do with my cache.

 

Sometimes it copies files with no problem sometimes, it does not.

 

I got 2 shares, and I am transferring BIG files from one to another using Krusader. I am not sure but maybe when I send the files from one share to another it seems that Krusader does it instantly. But maybe that is not the case? Maybe it is still working in the background? The reason I am asking is because my Cache seems to always be full (1TB) or close to full. 

 

The problem that I have is when I try to unrar a big file, it many times give me the "No Space" message.

 

Can you help me please?

 

 

 

 

tower-diagnostics-20200401-2235.zip 160.48 kB · 1 download

kinda sounds like you have a cache drive that fills up? raid1 has a max size of 250gb unraid will ignore anything after, and it will only use 1 cache drive (the primary cache drive) and ignore the rest. once the primary cache drive goes beyond 250gb in size, unraid can no longer write to the array cause the cache drive is "full" if you manually move the contents from the cache drive to the array it'll start working again. its a stupid bug. this is why i moved away from unraid and went back to windows based file servers. i got tired of issues like this. 

[SopraNo3]

Hi,

Thank you for the answer. Even when I manual press "Move" to move the files from the Cache, it never seems to go to more than 50GB Capacity. No idea why.

[trurl]

2 hours ago, burn0u7 said:

kinda sounds like you have a cache drive that fills up? raid1 has a max size of 250gb unraid will ignore anything after, and it will only use 1 cache drive (the primary cache drive) and ignore the rest. once the primary cache drive goes beyond 250gb in size, unraid can no longer write to the array cause the cache drive is "full" if you manually move the contents from the cache drive to the array it'll start working again. its a stupid bug. this is why i moved away from unraid and went back to windows based file servers. i got tired of issues like this. 

I see what you did earlier in this thread, and your explanation of it is a little off.

 

@SopraNo3, you can ignore all that. You don't even have multiple cache disks.

 

34 minutes ago, SopraNo3 said:

Thank you for the answer. Even when I manual press "Move" to move the files from the Cache, it never seems to go to more than 50GB Capacity.

Set your system share to cache-only for now. That way mover will ignore it and we can work on getting your other share moved to the array.

 

You have 2 shares that look like they need to be moved to the array.

 

One of these shares is anonymized in your diagnostics as B----p. You currently have that share set to cache-no. Move ignores cache-no and cache-only shares. You must set that share to cache-yes for mover to move it to the array.

 

The other share is anonymized as D--------r. That share is cache-yes, so mover should move it to the array.

 

So, to summarize:

 

Set system share to cache-only.

Set those other 2 shares I mentioned to cache-yes.

Run mover.

 

When it completes post new diagnostics.

[SopraNo3]

19 minutes ago, trurl said:

 

 

Set system share to cache-only.

Set those other 2 shares I mentioned to cache-yes.

Run mover.

 

When it completes post new diagnostics.

Hello, Thank you for taking the time to help out.

 

Ok, I did what you said and I will wait until it is over. Then post diagnostics.

 

My question is: When I move files from one share to another, does it use the cache disk? Because I do that a lot. I am using Krusader and although it shows the move being done immediately I get the feeling that work is being done on the background, using up all the cache for many hours maybe a day. (I might move close to 1TB per time).

 

Thank you.

 

 

[SopraNo3]

As you can see there is a lot of activity going on on the disks, most probably due to the moving of all those files using krusader. (I am assuming, I am by no means an expert).

 

I do not even know if the cache is being used during all those moves.

[trurl]

Just now, SopraNo3 said:

My question is: When I move files from one share to another, does it use the cache disk? Because I do that a lot. I am using Krusader and although it shows the move being done immediately I get the feeling that work is being done on the background, using up all the cache for many hours maybe a day. (I might move close to 1TB per time).

Did you read the link I gave you earlier?

 

Ideally, it would consider the settings of the destination user share when writing a file. If that destination user share was cached, then it would write to cache.

 

But as explained in #2 below  

On 2/20/2019 at 10:06 AM, trurl said:

2 surprising results when managing files in Krusader, Midnight Commander, command line, etc.

1. If you mix user shares and disks when moving/copying files, you can actually lose data, because Linux doesn't realize that the source path and the destination path might actually be the same file, and so tries to overwrite what it is trying to read. This is often referred to as the User Share Copy Bug.
2. The Linux command for Move and Rename are the same, mv. So when you try to move files from one user share to another, Linux will often simply rename the files so they have a different path on the same disk, in violation of any user share settings such as included disks. The workaround for this is to copy from source to destination, so that new files get created following the user share settings, then deleting from the source.

If you need me to clarify #2 above, let me know.

 

The take-away from all that is you must copy from source to destination, then delete from the source, instead of moving, in order to get it to consider the user share settings.

[trurl]

3 minutes ago, SopraNo3 said:

As you can see there is a lot of activity going on on the disks, most probably due to the moving of all those files using krusader. (I am assuming, I am by no means an expert).

 

I do not even know if the cache is being used during all those moves.

Is this screenshot after you started mover?

[SopraNo3]

2 minutes ago, trurl said:

 

 

The take-away from all that is you must copy from source to destination, then delete from the source, instead of moving, in order to get it to consider the user share settings.

Hello yes I read it but did not understand it at the time. Now I do. Instead of saying to Krusader MOVE file I should say COPY and then go back to the share where the original file is and delete it. Or another solution is to download the files I want on that particular share to begin with and avoid all this. Right? Because the problem now is that when this happens I cannot even unrar some files without getting the "Not enough space" message in windows.

[SopraNo3]

2 minutes ago, trurl said:

Is this screenshot after you started mover?

 Yes, Indeed it is. 

[trurl]

3 minutes ago, SopraNo3 said:

another solution is to download the files I want on that particular share to begin with and avoid all this. Right? Because the problem now is that when this happens I cannot even unrar some files without getting the "Not enough space" message in windows.

You might reconsider how you are using cache. There is no requirement to use cache for any user share writes. Most of the writes to my server are scheduled backups and queued downloads. I don't care if they take a little longer since I am not waiting on them. They go directly to the array where they are already protected by parity and they don't have to be moved.

 

Since you mention unrar, I assume you are using some dockers or something for downloading, etc. Depending on what you are using, it may be possible to get your applications to put things in their final destination instead of you having to get involved with Krusader, etc. Download to one share and unrar to another, for example. And each of those shares can have different settings on whether or not they use cache. Maybe the final destination wouldn't use cache and things would end up on the array.

 

And, of course, the whole point of cache and mover is for things to wind up on the array anyway. Since you are asking about whether or not cache is used in various situations, I think maybe you don't really understand how cache is used generally.

[SopraNo3]

I won't lie. I do not really know how it works.

 

But here is the scenario I want:

 

Once I find my files that I want to download, I insert them intro transmission. (I do not mind if that is not done automatically.) Then when they are done, I want to keep seeding them but if they are rar files I would like to unrar them, in a different location so the seeding location can remain untouched. I do not mind if the location is on a different share (This is what I was doing up to now, which I think it is wrong. I did this in the beginning when I thought that moving from one share to another would be the same like you move a Folder from an HDD to another folder on the SAME HDD. Stupid me:) ) or not. I think that being on the same share will save me from all this trouble. Am I right? Sorry for sounding like a complete newbie but I am one!

 

Edited April 1, 2020 by SopraNo3

[trurl]

How are you doing the unrar? Manually at the command line? (unrar from Nerd Pack?)

 

Builtin to Krusader?

 

[trurl]

4 hours ago, SopraNo3 said:

if they are rar files I would like to unrar them, in a different location so the seeding location can remain untouched. I do not mind if the location is on a different share (This is what I was doing up to now, which I think it is wrong. I did this in the beginning when I thought that moving from one share to another would be the same like you move a Folder from an HDD to another folder on the SAME HDD.

You say "moving" from one share to another but do you really want to just unrar and have the result go to a different share? All within Krusader?

 

I don't do anything like that myself. Only played with Krusader a little since I use the builtin Midnight Commander for file management.

 

A little googling though seems like Krusader will let you do that.

 

Basically, though, the way this and many other things will work in Unraid depends entirely on the settings for the destination user share. If it is set to cache-yes, then it will be written to cache and then moved to the array at the scheduled mover time. But Krusader (or anything else really) doesn't concern itself with the disks or cache or mover, it just knows about the paths and if those paths are user shares, Unraid takes care of which disks are involved based on the settings of each user share, and Unraid takes care of moving files from cache to array at the scheduled mover time.

[SopraNo3]

Actually, preferably I would like to not use Krusader at all. But copying from one share to another using my PC and Windows, involves the PC in the chain which takes up many resources. So I use Krusader to do that.

 

My steps up to now where:

 

1. Download the files (Lets assume they are rar) - they are 50 to 100GB each

2. Unrar them in the same directory.

3. Let the rared file in that directory for seeding.

4. Move the unrared file to the other share that is used as storage for finished files only.

5. When i decide that i have done enough seeding I delete the original file.

 

The problem was that if Unrar multiple files together and also at the same time that the unrar is taking place I also move some of the finished files, I would get the "No more space on disk" message which led me to think that something is wrong with the "Buffering" (I do not know if I can call it that). Maybe the files being moved and the ones being unrared (if it works like that) are taking up more space than the entire cache disk? No idea if what I am saying makes sense.

 

So what I thought to do is send the Downloaded files on the Final Destination share, so when I unrar them I won't have to move them to a different share. Just a different folder of the same share. This will help right? And the result will be the same.

 

Correct me if you are not bored and I am wrong

 

Mover finished by the way and now it shows 970GB free! Wow - You were right.

 

Here are my new diagnostics:

 

 

tower-diagnostics-20200402-1231.zip

[trurl]

Here is what makes sense to me. I don't know if it will work for you or not.

 

Download to some share where you will keep seeding. You need this share to not be cached (cache-no), since if you are keeping a lot of seeds going you don't want them to fill cache, and if they are seeding they can't be moved from cache since mover can't move open files. And you probably don't really need the speed of cache for torrents anyway. This is basically what I do myself with transmission.

 

Then Unrar to a different share. That share can use cache (cache-yes), which will make the extractions faster because it will be reading from the share where the seeds are (on the array) and writing to the cached share, so accessing different drives for the read and the write, and writing to the faster cache. Later the extracted files would be moved to the array at the scheduled mover time.

[trurl]

As you noted, you have plenty of room on cache now. What you would really like to do is go ahead and get the system share moved completely to cache, and get those other 2 shares we noted earlier completely moved to the array.

 

In order to get system share moved completely to cache, you will have to disable dockers (Settings - Docker), set system share to cache-prefer, and run mover.

 

Probably those other 2 shares didn't get moved completely to the array because they had open files, possibly transmission or similar had them open. When you run mover with docker disabled those should get moved too.

 

Try that and think about how you want this to work going forward.

 

 

[SopraNo3]

Regarding the first solution you recommend, I will need to find out how i can tell KRUSADER to unrar to a different share because up to now i was unraring in the same folder on the same share. You are right this makes a lot of sense! Let's see if I can do it.

 

2 hours ago, trurl said:

As you noted, you have plenty of room on cache now. What you would really like to do is go ahead and get the system share moved completely to cache, and get those other 2 shares we noted earlier completely moved to the array.

 

In order to get system share moved completely to cache, you will have to disable dockers (Settings - Docker), set system share to cache-prefer, and run mover.

 

Probably those other 2 shares didn't get moved completely to the array because they had open files, possibly transmission or similar had them open. When you run mover with docker disabled those should get moved too.

 

Try that and think about how you want this to work going forward.

 

 

Regarding moving the system share: What is the benefit of moving the system share to the cache? I thought that my system share was on the USB Flashdrive with the OS. Am I wrong? Also how safe is it to do what you are suggesting?

 

Thank you once again! 

 

[itimpi]

9 minutes ago, SopraNo3 said:

Regarding moving the system share: What is the benefit of moving the system share to the cache?

The normal reason for wanting the system share to be on the cache is to maximise performance of dockers and/or VMs.

10 minutes ago, SopraNo3 said:

I thought that my system share was on the USB Flashdrive with the OS. Am I wrong?

There are no shares on the flash drive (only configuration information).

[SopraNo3]

Thank you very much! noted. How about the safety question? How safe is it to try that?

[trurl]

Just follow the procedure I already gave here:

2 hours ago, SopraNo3 said:

In order to get system share moved completely to cache, you will have to disable dockers (Settings - Docker), set system share to cache-prefer, and run mover.

It is totally safe, in fact you could just delete it since docker image is easily recreated. Sometimes I recommend deleting it when a user has messed up their docker image by filling it or they have made it too large, but yours seems OK.

[SopraNo3]

On 4/2/2020 at 9:47 PM, trurl said:

Just follow the procedure I already gave here:

It is totally safe, in fact you could just delete it since docker image is easily recreated. Sometimes I recommend deleting it when a user has messed up their docker image by filling it or they have made it too large, but yours seems OK.

 

I did everything you said! Everything is working smoothly now! Thank you so much!

[SopraNo3]

Hello,

 

Everything seems to be working fine but i got these errors:

 

are they worrying?

 

[trurl]

The first one is telling you exactly what to do. Here is a FAQ with more details:

 

The second one is the problem we supposedly already fixed in this thread just a few posts above. Post new diagnostics so we can get a better idea what you might have done.

 

The third one is telling you exactly what to do.

 

No idea about the fourth one. Maybe with diagnostics someone else will have an idea.

 

The fifth one is telling you exactly what to do.