Unraid RC6: ZFS Pool "Unmountable: Unsupported or no file system"

[Chriexpe]

Out of nowhere the WebUI started crashing and the page would always go blank only coming back for a few seconds after refreshing the page many times, and I couldn't get the diagnostics because the page went blank in the middle of the process (through ssh it reported that there was no space left), but one thing that I've found odd is that Dockers kept working just fine.

The only way I found to get it working again was deleting disks.cfg from /boot/config and rebooting (and that was how I was able to generate the diagnostics file attached below), but after some time the problem always goes back.

So I'm pretty sure I've lost all my files, so before formatting them, is there any other useful data that I can attach here to prevent this from happening to me or other users? Before these problems begun I was trying to get Filerun (it's like Nexcloud) working with my ZFS volume (/mnt/zfs) as I had it working before reinstalling that docker, but it (filerun) couldn't recognize the files there, and at some point I tried running the docker on privileged status, I guess this might be the reason why my ZFS pool got corrupted.

Also, I had 5 Datasets in my ZFS pool, but now there is only 3 left with just a few files (but these where created by other dockers).

This ZFS pool was created with ZFS Master and Unassigned Devices Plugin (following SpaceInvaders video) and later on 6.12 I just straight up added to the pool and it was working just fine until now.

 

chriexpe.server-diagnostics-20230526-1001.zip

[Chriexpe]

Also on Main page my SSD (the only array device) and the pool keeps constantly showing and disappearing from there lol (after this webgui becomes inaccessible, it doesn't load).

[JorgeB]

Diags are before array start, cannot see what the error is, I do see the pool is already online, pool cannot be online before Unraid tries to import it, export the pool using the command line:

zpool export zfs

start array and post new ones.

[Chriexpe]

12 minutes ago, JorgeB said:

Diags are before array start, cannot see what the error is, I do see the pool is already online, pool cannot be online before Unraid tries to import it, export the pool using the command line:

zpool export zfs

start array and post new ones.

Ok, I waited for the array to start (and all containers go up) and attached the diag file.

I did the command before and after starting the pool and got the same response:

cannot open 'zfs': no such pool

 

chriexpe.server-diagnostics-20230526-1226.zip

[JorgeB]

5 minutes ago, Chriexpe said:

I did the command before and after starting the pool and got the same response:

cannot open 'zfs': no such pool

Yes, that was my bad, pool is available to import, not online, stop array, unassign all pool devices, start array, stop array, re-assign all pool devices, post new diags.

[Chriexpe]

26 minutes ago, JorgeB said:

Yes, that was my bad, pool is available to import, not online, stop array, unassign all pool devices, start array, stop array, re-assign all pool devices, post new diags.

Well, it's stil ummountable, before doing this I also changed default file system to ZFS on Disk Settings and even tried with different pool names, but got the same results.

chriexpe.server-diagnostics-20230526-1307.zip

[JorgeB]

Most likely you have a corrupt pool, post the output of

zpool import zfs

 

[Chriexpe]

19 minutes ago, JorgeB said:

Most likely you have a corrupt pool, post the output of

zpool import zfs

 

RIP

[email protected]:~# zpool import zfs
cannot import 'zfs': insufficient replicas
        Destroy and re-create the pool from
        a backup source.

 

[JorgeB]

10 minutes ago, Chriexpe said:

RIP

 

Looks like it, kind of strange that the before importing it shows all pool devices available and online:

pool: zfs
     id: 15271653718495853080
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

    zfs         ONLINE
      raidz1-0  ONLINE
        sdd     ONLINE
        sdc     ONLINE
        sdb     ONLINE

 

The are a few import options you could try but assuming there's a backup best to just recreate and restore the pool.

 

[Chriexpe]

6 minutes ago, JorgeB said:

 

Looks like it, kind of strange that the before importing it shows all pool devices available and online:

pool: zfs
     id: 15271653718495853080
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

    zfs         ONLINE
      raidz1-0  ONLINE
        sdd     ONLINE
        sdc     ONLINE
        sdb     ONLINE

 

The are a few import options you could try but assuming there's a backup best to just recreate and restore the pool.

 

I've never done any backup of it, is there any other way to import it?

[Chriexpe]

Also it's odd that if I type just zpool import it output exactly what you've sent
[email protected]:~# zpool import
 

   pool: zfs
     id: 15271653718495853080
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

        zfs         ONLINE
          raidz1-0  ONLINE
            sdd     ONLINE
            sdc     ONLINE
            sdb     ONLINE

 

[JorgeB]

1 minute ago, Chriexpe said:

Also it's odd that if I type just zpool import it output exactly what you've sent

That's normal, I see that because the output of that command comes with the diags.

 

There's a few options you can try, 1st one is to import in readonly mode:

 

zpool import -o readonly=on zfs

 

[Chriexpe]

5 hours ago, JorgeB said:

That's normal, I see that because the output of that command comes with the diags.

 

There's a few options you can try, 1st one is to import in readonly mode:

 

zpool import -o readonly=on zfs

 

Yup it did work, I was trying with -f and -F flags but had no success, now even tho unraid still reports the pool as unmountable I can access all files inside it, zpool status gives this result:
 

pool: zfs
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P
  scan: scrub repaired 0B in 04:41:10 with 0 errors on Mon Feb 27 16:58:43 2023
config:

        NAME        STATE     READ WRITE CKSUM
        zfs         ONLINE       0     0     0
          raidz1-0  ONLINE       0     0     0
            sdd     ONLINE       0     0     1
            sdc     ONLINE       0     0     0
            sdb     ONLINE       0     0     0

errors: No known data errors

I created this diag file after importing this pool.

chriexpe.server-diagnostics-20230526-1603.zip

Edited May 26 by Chriexpe

[JorgeB]

Make sure you backup anything important that you can then try exporting and re-importing the pool in read/write mode now, if it still fails to import best to recreate.

[Chriexpe]

14 hours ago, JorgeB said:

Make sure you backup anything important that you can then try exporting and re-importing the pool in read/write mode now, if it still fails to import best to recreate.

Thanks! Unfortunately I couldn't import my pool so I needed to format all disks, but thanks to you I was able to save the files.

I decided to use the GUI to create this new ZFS pool and yeah, definitely Unraid need some polishing with it, for example a separate button to manage the pool instead of clicking in the main disk, or after changing the filesystem and creating the ZFS pool warn the user and format the pool (instead of just saying that the pool is unmountable like my case), and automatically add the datasets to shares.

Edited May 27 by Chriexpe