Neldonado

I agree that I think it's connection related. Fact is I was in there moving drives around just hours before, and both of these drives were on different PSU's and different SAS breakout cables. I purchased a few replacement cables and will try to put the SSD's on the mobo or remove them entirely from the equation. Can you recommend the safest method for me to rebuild the parity and data disk?

I will try a new SAS breakout cable for that sds drive, or see if I have room to plug directly into the mobo. Do SSD's not play nice? Is there anything you can see that would tell you what caused these drives to disable in the first place? just loose cables? What's the safest way for me to go about rebuilding the contents assuming it's all there? thank you so much for all the time helping me.

OK, here is the 2nd drive and my diagnostics after starting the array! H7280A520SUN8.0T_001701PVZUGV_VLKVZUGV_35000cca260da2f70-20230308-0332.txt skynet-diagnostics-20230308-1134.zip

Here is the parity drive that failed. The data disk is still testing, and it looks like it's got another 4-5hrs until it's complete. ST10000NM0226_ZA290T3P0000C905MDCF_35000c500a6c8fc23-20230307-1812.txt

how long do the tests usually take? One of the drives has been at 100% for a few hours now but not complete. The second is at 67%... I started them both over 12hours ago. I also noticed this in the logs while waiting. The array is stopped.

I've posted my docs and extended tests for my drives. Both appear to be fine from what I can tell. I stopped my array after getting the diagnostics. I am not sure if leaving it on is a good idea until I know what's going on and how to fix. skynet-diagnostics-20230306-2345.zip ST10000NM0226_ZA290T3P0000C905MDCF_35000c500a6c8fc23-20230306-1532.txt H7280A520SUN8.0T_001701PVZUGV_VLKVZUGV_35000cca260da2f70-20230306-1532.txt

Which order of these should I go? Do tests, reboot, start array, post diags, and after that fix cache? Am I starting the array normally or in maintenance mode?

I will run a long smart test for both drives but I agree it looks more power/ connection related. How would I know if I fixed the problem by reseating all the drives? If I unassign, start, stop, and reassign the cache drives will I lose any data? how do I fix the disabled drives without losing any data (assuming smart tests come back ok)?

I started the day by moving my appdata, system, and domain folders off my cache pool onto the array so that I could upgrade to bigger cache drives. I moved the two 120gb ssd's into my DAS, and added two new 256gb ssds into my server in their place. I booted up, selected the new ssd's as the new disks for my appdata cache pool and hit the mover. Everything was fine. Then some time later my 2nd parity drive ended up being disabled. Mar 5 16:00:08 Skynet kernel: critical target error, dev sdy, sector 230862839 op 0x3:(DISCARD) flags 0x800 phys_seg 1 prio class 0 Mar 5 16:00:08 Skynet kernel: BTRFS warning (device sdy1): failed to trim 1 device(s), last error -121 Mar 5 16:00:08 Skynet root: /etc/libvirt: 920.8 MiB (965480448 bytes) trimmed on /dev/loop3 Mar 5 16:00:08 Skynet root: /var/lib/docker: 21.8 GiB (23369732096 bytes) trimmed on /dev/loop2 Mar 5 16:00:08 Skynet root: /mnt/skynetcache: 339.6 GiB (364687364096 bytes) trimmed on /dev/sdb1 Mar 5 16:00:13 Skynet sSMTP[18250]: Creating SSL connection to host Mar 5 16:00:13 Skynet sSMTP[18250]: SSL connection using TLS_AES_256_GCM_SHA384 Mar 5 16:00:15 Skynet sSMTP[18250]: Sent mail for [email protected] (221 2.0.0 closing connection i8-20020aa79088000000b00594235980e4sm5017473pfa.181 - gsmtp) uid=0 username=root outbytes=468 Mar 5 16:10:24 Skynet webGUI: Successful login user root from 192.168.1.228 Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Mar 5 16:15:30 Skynet kernel: sd 15:0:17:0: device_block, handle(0x001c) Mar 5 16:15:30 Skynet kernel: sd 15:0:18:0: device_block, handle(0x001d) Mar 5 16:15:30 Skynet kernel: sd 15:0:17:0: [sdu] tag#3035 UNKNOWN(0x2003) Result: hostbyte=0x0e driverbyte=DRIVER_OK cmd_age=0s Mar 5 16:15:30 Skynet kernel: sd 15:0:17:0: [sdu] tag#3035 CDB: opcode=0x2a 2a 00 0f 6d 59 e5 00 00 03 00 Mar 5 16:15:30 Skynet kernel: I/O error, dev sdu, sector 2070597416 op 0x1:(WRITE) flags 0x0 phys_seg 3 prio class 0 Mar 5 16:15:30 Skynet kernel: md: disk29 write error, sector=2070597352 Mar 5 16:15:30 Skynet kernel: md: disk29 write error, sector=2070597360 Mar 5 16:15:30 Skynet kernel: md: disk29 write error, sector=2070597368 Mar 5 16:15:30 Skynet kernel: sd 15:0:17:0: [sdu] tag#3034 UNKNOWN(0x2003) Result: hostbyte=0x0e driverbyte=DRIVER_OK cmd_age=0s Mar 5 16:15:30 Skynet kernel: sd 15:0:17:0: [sdu] tag#3034 CDB: opcode=0x2a 2a 00 00 5d c4 30 00 00 04 00 Mar 5 16:15:30 Skynet kernel: I/O error, dev sdu, sector 49160576 op 0x1:(WRITE) flags 0x0 phys_seg 4 prio class 0 Mar 5 16:15:30 Skynet kernel: md: disk29 write error, sector=49160512 Mar 5 16:15:30 Skynet kernel: md: disk29 write error, sector=49160520 Mar 5 16:15:30 Skynet kernel: md: disk29 write error, sector=49160528 Mar 5 16:15:30 Skynet kernel: md: disk29 write error, sector=49160536 Mar 5 16:15:31 Skynet kernel: sd 15:0:17:0: device_unblock and setting to running, handle(0x001c) Mar 5 16:15:31 Skynet kernel: sd 15:0:17:0: Power-on or device reset occurred Mar 5 16:15:31 Skynet kernel: sd 15:0:18:0: device_unblock and setting to running, handle(0x001d) Mar 5 16:15:31 Skynet kernel: sd 15:0:18:0: Power-on or device reset occurred Mar 5 16:15:33 Skynet flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup update Mar 5 16:21:04 Skynet rpc.mountd[6271]: v4.2 client detached: 0x6f4ca8e26404f5fe from "192.168.1.48:768" after seeing the logs I assumed I may have bumped a cable loose or something like that, so I stopped the array, shut it down, and reseated all the drives. I booted back up and was presented with no errors but the drive was still disabled. So I began to look up the process to re-enable the drive, and roughly 20-30 minutes later a data disk throws up an error and is disabled. I checked the logs and this is all I saw: so I panic and decide to stop the array and shut it down. The array stopped, or so I thought, and then I saw at the bottom it said Array Stopping•Sync filesystems... after around 20 minutes of it just completely locked up like this I did a hard shut down. I pulled both the server and das out and reseated every cable and even moved some drives around from the das to server and server to das just for more trouble shooting help. I booted everything back up and did a smart test on the drives (attached) ST1 is the parity drive, and H72 is the data disk. I also just noticed that my two new cache drives (that holds my appdata, domains, and system) are showing up as incorrect. I've uploaded my diagnostics... At this point I am totally lost at what I should be doing. I think I have two issues here, 1) the two disabled disks, and 2) the cache drives reporting as "wrong" when they are not. Any help is appreciated. skynet-diagnostics-20230306-0325.zip

any reason I couldn't map this to my appdata folder? I have a 2 ssd cache pool where nothing but my appdata sits. Wondering if that's a bad idea due to read/write or its fine?

How do you run the sync server? unraid?

I recently noticed my scanned documents are just sitting in the file queue. https://i.hah.rip/j2m.png I use a scanning app on my phone, and send them to a webdav that moves them to the consume folder. They are immediately picked up by paperless and but then they just sit in the queued state. I don't get any errors from the log, so I am a bit lost of where to even look? I am running this on unraid, from ghcr.io/paperless-ngx/paperless-ngx Here is my log after a fresh restart and you can see it see's the two documents in my folder. https://paste.hah.rip/?703a6fc41ea425aa#HHECEbTDQzwmHEAq1Hk9pEy8RgskBTgyHinYH5ygicNb

Hello! I am trying to upgrade my nextcloud versions and keep getting this error after typing docker exec -it nextcloud occ upgrade I get this error: root@Skynet:~# docker exec -it nextcloud occ upgrade Nextcloud or one of the apps require upgrade - only a limited number of commands are available You may use your browser or the occ upgrade command to do the upgrade Set log level to debug Updating database schema An unhandled exception has been thrown: Error: Class 'Doctrine\DBAL\Types\Types' not found in /config/www/nextcloud/core/Migrations/Version21000Date20210309185126.php:49 Stack trace: #0 /config/www/nextcloud/lib/private/DB/MigrationService.php(478): OC\Core\Migrations\Version21000Date20210309185126->changeSchema() #1 /config/www/nextcloud/lib/private/DB/MigrationService.php(414): OC\DB\MigrationService->executeStep() #2 /config/www/nextcloud/lib/private/Updater.php(298): OC\DB\MigrationService->migrate() #3 /config/www/nextcloud/lib/private/Updater.php(244): OC\Updater->doCoreUpgrade() #4 /config/www/nextcloud/lib/private/Updater.php(131): OC\Updater->doUpgrade() #5 /config/www/nextcloud/core/Command/Upgrade.php(257): OC\Updater->upgrade() #6 /config/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OC\Core\Command\Upgrade->execute() #7 /config/www/nextcloud/3rdparty/symfony/console/Application.php(915): Symfony\Component\Console\Command\Command->run() #8 /config/www/nextcloud/3rdparty/symfony/console/Application.php(272): Symfony\Component\Console\Application->doRunCommand() #9 /config/www/nextcloud/3rdparty/symfony/console/Application.php(148): Symfony\Component\Console\Application->doRun() #10 /config/www/nextcloud/lib/private/Console/Application.php(214): Symfony\Component\Console\Application->run() #11 /config/www/nextcloud/console.php(99): OC\Console\Application->run() #12 /config/www/nextcloud/occ(11): require_once('/config/www/nex...') #13 {main}root@Skynet:~# I am not really sure what's going on. My nextcloud instance is kind of stuck in this limbo now and I can't find anyone with an issue similar to mine. Any help would be appreciated.

I reinstalled it and deleted all the files, it worked 100% for 3-4 hours and now some of my most recent clips are not working. It seems inconsistent with the ones I get errors with and the one's I don't. Only thing I can think is that I am using a cache with the video share.. would that affect anything?

So I went and deleted the DB, did another scan, and videos older than a specific date wont play. Some of these are videos that I had shared links to, so I know they worked at one point, and they are all created and stored in the same folder by Geforce experience. I can play all of the files straight from the folder, but on fireshare only newer videos will play. Any suggestions besides a full delete/ reinstall? I'd hate to lose all my links I've shared. I should note I can go back a week, which is when I had to restore my appdata. Maybe something got messed up with that? Would the fireshare process folder be not lined up with my db?

I might give this a shot. Recent videos work but my older ones stopped, and I recently had to restore my appdata so maybe that messed something up?

Started running into this issue where all my videos show this error when I go to play them. The directory is correct and all the files are here in MP4 Format, so I am not sure what's going on. Any suggestions?

Here are my diagnostics! skynet-diagnostics-20220925-0853.zip

I know this is a pretty old thread, but I have run into the same issue. I rebooted, and was able to mount my drive, however when I go to unmount it and mount it again later I get the same error. Is there a better fix from this happening? Sep 24 20:43:46 Skynet unassigned.devices: Mount of 'sda1' failed: 'fuse: mountpoint is not empty fuse: if you are sure this is safe, use the 'nonempty' mount option FUSE exfat 1.3.0 ' Sep 24 20:43:46 Skynet unassigned.devices: PHP Warning: rmdir(/mnt/disks/133009479092): Directory not empty in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 1375 Sep 24 20:43:46 Skynet unassigned.devices: Partition '133009479092' cannot be mounted.

Yeah, everything looks good so far. I am hoping it is ram related, that would be an easy fix. I had some Corruption issues a few weeks ago too. (you can see my other posts) however this isn't as bad because I don't think any data was lost, and I am pretty sure it was bad drive related.

Is it ok to let parity check finish? I will have to remove one of my drives/ pcie card to put a video card in.

Is it possible to do that through the unraid GUI? I don't have a video out on my MOBO

I found some of my dockers not working this morning, when I go to check the unraid logs it shows these errors. I found a post by Squid where he suggested deleting the docker image and reinstalling the dockers. I did this following his steps, and when I went to start them up again the docker services wouldn't start. Not know what else I could do I decided to do a clean reboot of the server. When the server came back online a parity check was started, and all of my dockers started and were running without error. On one hand I am happy everything is working, but I would like help to figure out why this happened in the first place, so that I could avoid it. I did download my diagnostics before rebooting, so hopefully that helps. 03:09:19 Skynet kernel: BTRFS info (device loop2): no csum found for inode 8016 start 0 Aug 13 03:09:19 Skynet kernel: BTRFS warning (device loop2): csum failed root 2905 ino 8016 off 0 csum 0xb0d03f8f expected csum 0x00000000 mirror 1 Aug 13 03:09:19 Skynet kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 58, rd 0, flush 0, corrupt 43796, gen 0 Aug 13 03:09:19 Skynet kernel: BTRFS error (device loop2): parent transid verify failed on 8222932992 wanted 1195432 found 1195419 Aug 13 03:09:19 Skynet kernel: BTRFS error (device loop2): parent transid verify failed on 8222932992 wanted 1195432 found 1195419 Aug 13 03:09:19 Skynet kernel: BTRFS info (device loop2): no csum found for inode 8016 start 0 Aug 13 03:09:19 Skynet kernel: BTRFS warning (device loop2): csum failed root 2905 ino 8016 off 0 csum 0xb0d03f8f expected csum 0x00000000 mirror 1 Aug 13 03:09:19 Skynet kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 58, rd 0, flush 0, corrupt 43797, gen 0 Aug 13 03:09:20 Skynet kernel: BTRFS info (device loop2): no csum found for inode 8016 start 0 skynet-diagnostics-20210813-0500.zip

I have moved the cables to different drives and they connect fine. This was my sys log after starting the array. https://paste.ubuntu.com/p/2wRZH3pMZN/

I added it back to the same slot.