Hello, I am looking for some help with my docker containers. I recently had some issues with my docker containers and decided to delete the image. After I deleted the image, some of the containers start and some done. If I manually start them, I get an error 403. I did some searching and I think it may have something to do with logs somehow but not too sure.

 

I have attached my diagnostics. Thanks in advance.

tower-diagnostics-20240727-1026.zip

*Update*: After posting the diagnostics I started the array and now the cache disks are saying "Unmountable: Unsupported or no file system"

Diags after array start please.

27 minutes ago, JorgeB said:

Diags after array start please.

My mistake, I couldn't remember to get them before or after array start. I attached them to this post. Thanks!

tower-diagnostics-20240727-1628.zip

Jul 27 13:23:57 Tower kernel: BTRFS info (device sdg1): bdev /dev/sdg1 errs: wr 14579960, rd 10433665, flush 48340, corrupt 1151794, gen 0
Jul 27 13:23:57 Tower kernel: BTRFS info (device sdg1): bdev /dev/sdf1 errs: wr 23172219, rd 57409, flush 546115, corrupt 3, gen 0

 

Pool is a mess, both devices dropped offline in the past, though the current issue is the log tree, see if this helps, but probably there will be other issues, type in the console:

 

btrfs rescue zero-log /dev/sdg1

 

Then re-start the array, if the pool mounts, run a correcting scrub and post the results.

Hello, I was able to restart the array and both drives mounted

 

Scrub returned the following:

 

UUID: f77815f4-da02-4c6a-a168-4f1d83b1ce1c Scrub started: Sun Jul 28 06:59:39 2024 Status: finished Duration: 0:07:51 Total to scrub: 409.95GiB Rate: 891.28MiB/s Error summary: no errors found

Reset the pool stats and keep monitoring for new errors:

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=700582

 

10 hours ago, JorgeB said:

Reset the pool stats and keep monitoring for new errors:

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=700582

 

Thanks! I followed the guide and reset the errors. FYI, before I reset the errors the output was:

root@Tower:~# btrfs dev stats /mnt/cache
[/dev/sdg1].write_io_errs    14579960
[/dev/sdg1].read_io_errs     10433665
[/dev/sdg1].flush_io_errs    48340
[/dev/sdg1].corruption_errs  1151794
[/dev/sdg1].generation_errs  0
[/dev/sdf1].write_io_errs    23172219
[/dev/sdf1].read_io_errs     57409
[/dev/sdf1].flush_io_errs    546115
[/dev/sdf1].corruption_errs  3
[/dev/sdf1].generation_errs  0

 

 

 

 

Side note, should I be scrubbing or balancing on some kind of schedule? This is the first time since using unraid (4-5 years) I ran into this problem. Thanks again!

Good idea to scrub once a month, but more important is to keep monitoring it.

Hello, I found more errors when I checked using the monitoring guide you provided. I'm thinking that this drive is failing. It is still in warranty since I only bought it in Jan 2023. 

 

I attempted to convert the cache to single mode and was able to mount the "good" drive. I tried to grab the appdata folder and back it up to my personal PC but I get a windows error that the network drive is unavailable. Is it OK to run dockers as normal while only having one cache disk since the other one will be removed and returned for warranty?

 

*EDIT* fix common problems says the single drive is now in read only mode.

tower-diagnostics-20240730-0918.zip

Edited July 30, 20241 yr by PrisonMike
diagnostics, update

Looks more like a filesystem issue to me, I would recommend backing up and reformatting the pool, then restore the data.

On 7/30/2024 at 10:32 AM, JorgeB said:

Looks more like a filesystem issue to me, I would recommend backing up and reformatting the pool, then restore the data.

 

 

Well, I'm about ready to pull my hair out. I cant get docker service to start so I can run Krusader, I cant get windows to play nicely so I can copy appdata folder to my desktop on my PC. Basically my appdata folder is trapped and cant be moved anywhere. Maybe there is some kind of permissions issue maybe I have never had this problem before with my unraid server and its very frustrating. Scrub and balance abort themselves when I try and start them. Nothing seems to be working.

Edited August 2, 20241 yr by PrisonMike

11 minutes ago, PrisonMike said:

Scrub and balance abort themselves when I try and start them

 

On 7/30/2024 at 3:32 PM, JorgeB said:

I would recommend backing up and reformatting the pool, then restore the data.

 

4 minutes ago, JorgeB said:

Scrub and balance abort themselves when I try and start them

 

Hmm, I see. Maybe I am going about backing up the pool in the wrong way, I also have the dynamix file manager plugin but that hangs on binhex-krusader when I try to copy the appdata from "/appdata" to "/mainshare" 

Edited August 2, 20241 yr by PrisonMike

You will need to try and copy what you can, may not be able to copy everything since the pool is corrupt.

2 hours ago, JorgeB said:

You will need to try and copy what you can, may not be able to copy everything since the pool is corrupt.

OK, I will try to copy each container by itself. Thanks for the heads up!

Hello, I have backed up what I can from the cache drive to my windows PC. However, I don't understand why I am not able to just remove the bad drive and replace it with a new one. I have two SSDs in the cache pool that should have been mirroring each other. Are both drives corrupted?

tower-diagnostics-20240803-1604.zip

It's a filesystem problem, not a device problem, so both are affected.