ZFS Snapshot destroy - dataset is busy - Docker

Not sure what the problem could be, but you could try send/receive that snapshot to a new dataset and the then run the container on that dataset, e.g.:

zfs snapshot cache-mirror/appdata@borgmatic
zfs send cache-mirror/appdata@borgmatic | zfs receive cache-mirror/borgmatic

 

Then run the container on that dataset instead, you don't need to point to the snapshot since the dataset will have the same data, so instead of:

/mnt/cache-mirror/appdata/.zfs/snapshot/borgmatic

use

/mnt/cache-mirror/borgmatic

 

Once the backup is done try deleting the dataset cache-mirror/borgmatic

 

 

 

 

Ok, tested the following.

 

Working:

• Create Snapshot
• Start Container
• Stop Container
• Destroy Snapshot

 

Not Working:

• Create Snapshot
• Start Container
  • ls /mnt/cache-mirror/appdata/.zfs/snapshot/borgmatic
    • ls: /mnt/cache-mirror/appdata/.zfs/snapshot/borgmatic/: Symbolic link loop
• Stop Container
• Destroy Snapshot
  • cannot destroy snapshot cache-mirror/appdata@borgmatic: dataset is busy

 

 

Trying to fix the problem:

• Disable Docker
  • Still Unable to delete Snapshot

 

• Stop Array
• Start Array (still stopped docker)
  • Destroy is working

 

Looks like an process is stuck?

 

Next step i will try your suggestion

 

mmmh, i am afraid this is not what i want to achieve?

 

zfs send cache-mirror/appdata@borgmatic | zfs receive cache-mirror/borgmatic

 

The process is now running but i am afraid now it is copying all data from the snapshot time to a cache-mirror/borgmatic dataset?

I dont want to clone the complete folder. My appdata folder has 150GB ^^

 

This is contraproductive, the backup would only copy the delta and now i would clone the complete directory for every backup.

Still will test it, if it works this way, it still needs some time to copy the data

Edited June 13, 2023 by unr41dus3r

19 minutes ago, unr41dus3r said:

The process is now running but i am afraid now it is copying all data from the snapshot time to a cache-mirror/borgmatic dataset?

Yes, but unless I missed something you were also snapshotting the complete appdata folder before:

zfs snapshot cache-mirror/appdata@borgmatic

and than backing up the complete snapshot:

Quote

Try to backup /mnt/cache-mirror/appdata/.zfs/snapshot/borgmatic

Or did I misunderstood?

 

9 minutes ago, JorgeB said:

Yes, but unless I missed something you were also snapshotting the complete appdata folder before:

zfs snapshot cache-mirror/appdata@borgmatic

and than backing up the complete snapshot:

Or did I misunderstood?

 

 

Yeah i did a complete snapshot of the appdata folder but this means, the state is frozen and all new files are written additional.

 

With the send | recieve command it copys the complete frozen state into a new dataset.

So i have the cache-mirror/appdata dataset incl. snapshot and now a new dataset cache-mirror/borgmatic

 

I dont know how zfs exactly works with compression and dedup in this situation but now it uses 2 times the space.

Also it did a more or less copy operation when i checked the saturation of my SSDs.

 

Edit:

With this method, i dont recieve the error with the symlink loop / when trying to destroy that the dataset is busy.

 

But the problem is still its to much overhead as i understand.

 

Example:

• cache-mirror/appdata = 150GB
• cache-mirror/appdata@borgmatic = Snapshot has only some MB or maybe 1GB

 

• Now i start abackup of the snapshot (frozen state of the specific time)

 

When i issue the send | recieve command i need much more space and time/performance

• cache-mirror/appdata = 150GB
• cache-mirror/appdata@borgmatic = 1GB (for example)
• cache-mirror/borgmatic = 150GB

 

So i need double the space and it needs time to the copy data.

 

I hope it is understandable what i mean.

Edited June 13, 2023 by unr41dus3r

You could create a child dataset just for borgmatic, then snapshot and send/receive only that dataset.

25 minutes ago, JorgeB said:

You could create a child dataset just for borgmatic, then snapshot and send/receive only that dataset.

 

Could you explain what you mean with child dataset? I dont exactly know how to do this.

 

About the loop error message in an container, is this expected or is their something wrong?

On second though I think a zfs clone would be more efficient for what you need, assuming it still works:

zfs snapshot cache-mirror/appdata@borgmatic
zfs clone cache-mirror/appdata@borgmatic cache-mirror/borgmatic

 

This won't take any extra space unless source dataset is changed, same as a snapshot.

 

Then work with /mnt/cache-mirror/borgmatic and once done destroy the clone and the snapshot:

zfs destroy cache-mirror/borgmatic
zfs destroy cache-mirror/appdata@borgmatic

 

Another interesting find.

 

If i create a snapshot in cache-mirror@borgmatic and the snapshot dataset is stuck.

I recieve the error which i described in the RC7 thread when stopping the array/ rebooting

 

Jun 13 15:25:22 tower emhttpd: shcmd (9223): /usr/sbin/zpool export cache-mirror
Jun 13 15:25:22 tower root: cannot unmount '/mnt/cache-mirror': pool or dataset is busy
Jun 13 15:25:22 tower emhttpd: shcmd (9223): exit status: 1

 

 

If i create a snapshot in cache-mirror/appdata@borgmatic and the snapshot dataset is stuck.

It is possible to stop the array normally.

1 minute ago, JorgeB said:

On second though I think a zfs clone would be more efficient for what you need, assuming it still works:

zfs snapshot cache-mirror/appdata@borgmatic
zfs clone cache-mirror/appdata@borgmatic cache-mirror/borgmatic

 

This won't take any extra space unless source dataset is changed, same as a snapshot.

 

Then work with /mnt/cache-mirror/borgmatic and once done destroy the clone and the snapshot:

zfs destroy cache-mirror/borgmatic
zfs destroy cache-mirror/appdata@borgmatic

 

 

Thanks will try it later

4 hours ago, JorgeB said:

On second though I think a zfs clone would be more efficient for what you need, assuming it still works:

zfs snapshot cache-mirror/appdata@borgmatic
zfs clone cache-mirror/appdata@borgmatic cache-mirror/borgmatic

 

This won't take any extra space unless source dataset is changed, same as a snapshot.

 

Then work with /mnt/cache-mirror/borgmatic and once done destroy the clone and the snapshot:

zfs destroy cache-mirror/borgmatic
zfs destroy cache-mirror/appdata@borgmatic

 

 

This works great, this is the solution i was looking for!

Still a little bit strange the stuck dataset if you use a path in a container

Edited June 13, 2023 by unr41dus3r

I will close the issue, i have an solution, still dont know if this is an normal behavior.

Changed Status to Solved