Recycle Bin (vfs recycle) for SMB Shares


dlandon

794 posts in this topic Last Reply

Recommended Posts

22 minutes ago, tjb_altf4 said:

 

It seems the samba log is including records we're less interested in logging, and excluding what we are interested in?

Give me some examples.  I am doing some filtering because the UD log would get unwieldy.

Link to post
  • Replies 793
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

This is a plugin that will create a per share .Recycle.Bin folder on each smb share for files deleted on that share.  Built into samba is a module called "vfs recycle" that handles the deletion of fil

Shares were not displaying when "Log Deleted Files" was set to "No".  This has been fixed in the latest version.

No longer true.  I'm releasing a new version that will not show any remote mounted shares and the local recycle bin will not remove any files from the remote share recycle bins.  Only the local server

Posted Images

7 hours ago, dlandon said:

Give me some examples.  I am doing some filtering because the UD log would get unwieldy.

Understand.

Lets start from the top, I've cleared the log and excluded the games share that was pumping the steam cache entries.

Here a run through of deleting a file:

 

Initial state

image.png.d7e71668444c0eec70c1357875eb6c27.png

 

Created file to delete, then deleted

image.png.8c4c92579ff6e1dea4f56dff28258096.png

 

Share appears, no delete file log, .Recycle.Bin directory and deleted file appears

image.png.2fd8cb16f74ebffbab80c7df23ba492e.png

 

Emptied recycle bin, still no deleted file logged

 .Recycle.Bin directory + contents and GUI reference to share disappear as expected

image.png.ba2b050528ba25a24a84d6b0d147717d.png

 

Output of cat /var/log/samba/log.smbd

image.png.32613e08cbfe9d4fe58fdc9a93be2934.png

 

[2018/11/17 08:30:23.832866,  0] ../source3/modules/vfs_extd_audit.c:159(audit_mkdir)
  vfs_extd_audit: mkdir /mnt/user/software/.Recycle.Bin
[2018/11/17 08:30:23.833067,  0] ../source3/modules/vfs_extd_audit.c:159(audit_mkdir)
  vfs_extd_audit: mkdir /mnt/user/software/.Recycle.Bin/Kodi

 

Edited by tjb_altf4
Link to post
44 minutes ago, tjb_altf4 said:

Understand.

Lets start from the top, I've cleared the log and excluded the games share that was pumping the steam cache entries.

Here a run through of deleting a file:

The log.smbd is managed by the vfs_recycle module of samba.  I do not have any control over the log except to enable it and let vfs_recycle manage the entries.  I can check the settings for enabling the log, but I'm pretty sure it is right.  I have no explanation for the file deletion not getting logged.

 

What version of Unraid are you using?

Link to post
1 hour ago, dlandon said:

What version of Unraid are you using?

6.6.5

 

I think I may have spotted the issue:

/etc/samba/smb-shares.conf has .Recycle.Bin as an excluded directory on all shares.

recycle:exclude_dir = .Recycle.Bin

I don't think this is needed as it is named as the recycle repository 

recycle:repository = %P/.Recycle.Bin/

With the .Recycle.Bin directory excluded it won't pickup when a file is deleted from that directory (when it empties).

 

Edited by tjb_altf4
Link to post
21 minutes ago, tjb_altf4 said:

With the .Recycle.Bin directory excluded it won't pickup when a file is deleted from that directory (when it empties).

Correct.  I did that on purpose.  I don't think there is any value in logging deleted files from the .Recycle.Bin directory.

Link to post

I understand the rationale, but that means deleted files never get logged as being deleted as they aren't being logged as deleted when they go to the recycle bin folder.

Only deleted files that bypass recycle bin are logged as deleted.

 

Or am I misunderstanding the intent of the deleted files log?

Link to post
8 hours ago, tjb_altf4 said:

I understand the rationale, but that means deleted files never get logged as being deleted as they aren't being logged as deleted when they go to the recycle bin folder.

Only deleted files that bypass recycle bin are logged as deleted.

 

Or am I misunderstanding the intent of the deleted files log?

When a file is deleted from a SMB share, they are moved to the .Recycle.Bin and logged by the vfs_recycle module of SMB.  Deleting files from the .Recycle.Bin itself does not log a deleted file.  I don't understand why your file is not logging as deleted.  I'll review your post again to see if I can see something about your particular file causing a problem.

Link to post
2 hours ago, warham said:

Removed the plugin and reinstalled.  trying to open plugin just opens a blank page with the unraid gui at the top.  This started with 6.6.5.  Ideas to fix it?

You probably have a large .Recycle.Bin and it is taking a long time to determine the Recycle Bin size.  This may also have to spin up disks which can cause a delay.  Do you have the cache dirs plugin installed?

Link to post
7 hours ago, warham said:

dynamix cache dr but its disabled. I will let recycble bin sit for a while and It did eventually come up although it says its disabled.   ill play with it from there.  thank you.

I've updated the plugin to time out if it takes too long to calculate the recycle bin size so it doesn't hang so long.

Link to post
On 11/17/2018 at 7:47 AM, warham said:

dynamix cache dr but its disabled. I will let recycble bin sit for a while and It did eventually come up although it says its disabled.   ill play with it from there.  thank you.

@warham Can you update to the latest version (today) and then go to the recycle bin shares tab?  Let me know if there are any '?' on any share sizes.  A '?' is shown if it takes too long to get the recycle bin size.

 

My suspicion is that the delay is being caused by one or more shares having a very large number of files.  Probably from a downloader or torrent.  I want to get to the bottom of this because a large number of files going to the recycle bin is thrashing the recycle bin and the array.  The solution is to exclude the share or certain file types.  Having a large number of files in the recycle bin is difficult to navigate and unwieldy.

Link to post
10 minutes ago, dlandon said:

@warham Can you update to the latest version (today) and then go to the recycle bin shares tab?  Let me know if there are any '?' on any share sizes.  A '?' is shown if it takes too long to get the recycle bin size.

 

My suspicion is that the delay is being caused by one or more shares having a very large number of files.  Probably from a downloader or torrent.  I want to get to the bottom of this because a large number of files going to the recycle bin is thrashing the recycle bin and the array.  The solution is to exclude the share or certain file types.  Having a large number of files in the recycle bin is difficult to navigate and unwieldy.

Once it finally opened I emptied the trash so there is nothing in there, if I am looking at the location your asking about.   It seems to be working.  Maybe next time just don't let recycle build up so much?  Thanks for the reply.  

image.png

Edited by warham
Link to post

Getting an invalid url error trying to install the most recent update.

 

plugin: updating: recycle.bin.plg




plugin: downloading: "https://github.com/dlandon/recycle.bin/raw/master/recycle.bin-2018.11.18a.tgz" ... failed (Invalid URL / Server error response)
plugin: wget: "https://github.com/dlandon/recycle.bin/raw/master/recycle.bin-2018.11.18a.tgz" download failure (Invalid URL / Server error response)
DONE

 

Link to post

 

1 minute ago, wgstarks said:

Getting an invalid url error trying to install the most recent update.

 


plugin: updating: recycle.bin.plg




plugin: downloading: "https://github.com/dlandon/recycle.bin/raw/master/recycle.bin-2018.11.18a.tgz" ... failed (Invalid URL / Server error response)
plugin: wget: "https://github.com/dlandon/recycle.bin/raw/master/recycle.bin-2018.11.18a.tgz" download failure (Invalid URL / Server error response)
DONE

 

Just fixed it.

Edited by dlandon
Link to post
1 hour ago, warham said:

Once it finally opened I emptied the trash so there is nothing in there, if I am looking at the location your asking about.   It seems to be working.  Maybe next time just don't let recycle build up so much?  Thanks for the reply.  

image.png

Be sure to set "Remove Aged Files on Schedule?" to 'Yes' to be sure the recycle bin does not grow too large.

 

EDIT: A large recycle bin does not really have much value because it takes so long to find things.

Edited by dlandon
Link to post
  • 1 month later...
1 hour ago, 1812 said:

How long is it suppose to take for the recycle bin to load and unload at array start/stop. I feel like me is taking longer and longer...

Do you mean start and stop?  There really isn't anything to load and unload.

Link to post
1 hour ago, dlandon said:

Do you mean start and stop?  There really isn't anything to load and unload.

Yeah, when I boot up my server, it takes what feels like 1-2 minutes just loading the recycling bin, and the same stopping.

 

actually, looking at the log, I think it's just displaying that it is starting the recycle bin, does it,  but getting hung on auto-mounting  (and failing to mount) remote disks via unassigned devices... so disregard for now.

Link to post

Hello @dlandon

I am having some issues with this plugin on unRaid 6.7.0 rc2 (wasnt paying attention on rc1 sorry)

I run three unraid server all with the recycle bin plugin and a couple shares mapped (using unassigned devices) from and two each ill try and explain

the server names are this

 

Thor

Sif

Freya


Thor has two mapped shares /mnt/disks/Sif_Backup and /mnt/disks/Freya_Media
Freya has one mapped back to Thor with /mnt/disks/THOR_Media

the issue is if i delete files on Freya both Thor and Freya see the same recycle bin size. and vice versa if i delete on thor Freya see the files in its recycle bin, if the files reside on Thor and i empty on Freya both recycle bins empty

I have use unassigned devices set to no on all server and even set the shares mounted to be excluded

but both still show each others and combined recycle bin

can you take a look please and if you need anything else let em know

 

 

Link to post
3 hours ago, Can0nfan said:

Hello @dlandon

I am having some issues with this plugin on unRaid 6.7.0 rc2 (wasnt paying attention on rc1 sorry)

I run three unraid server all with the recycle bin plugin and a couple shares mapped (using unassigned devices) from and two each ill try and explain

the server names are this

 

Thor

Sif

Freya


Thor has two mapped shares /mnt/disks/Sif_Backup and /mnt/disks/Freya_Media
Freya has one mapped back to Thor with /mnt/disks/THOR_Media

the issue is if i delete files on Freya both Thor and Freya see the same recycle bin size. and vice versa if i delete on thor Freya see the files in its recycle bin, if the files reside on Thor and i empty on Freya both recycle bins empty

I have use unassigned devices set to no on all server and even set the shares mounted to be excluded

but both still show each others and combined recycle bin

can you take a look please and if you need anything else let em know

 

 

If you are sharing UD shares among the servers, the Recycle Bin on those shares will also show up on the other servers.

 

Can you show screen shots of the Recycle Bin settings and the 'Shares' tab on one of the servers?

Link to post
20 hours ago, dlandon said:

If you are sharing UD shares among the servers, the Recycle Bin on those shares will also show up on the other servers.

 

Can you show screen shots of the Recycle Bin settings and the 'Shares' tab on one of the servers?

Hi I would love to but i downgraded one of my servers back to 6.6.6 and now i cant mount any shares from 6.7.0 rc2 and the recyle bin plugin isnt loading at all on the one i downgraded. im going to remove unassigned devices as that seems to be what hanging Recyclebin up ill try and get you screenshots once i duplicated it

Link to post
  • dlandon changed the title to Recycle Bin (vfs recycle) for SMB Shares

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.