nexusmaniac

Members
  • Posts

    251
  • Joined

  • Last visited

Everything posted by nexusmaniac

  1. Brill! Thank you ๐Ÿ™‡โ€โ™‚๏ธ
  2. It's happened again ๐Ÿ‘€ We can rule out csrf_token as the cause too, that's another symptom of the problem, it would seem ๐Ÿ˜ฐ Typing diagnostics command runs but it's very upset (output & ZIP attached... It's just empty files because nothing can be written to the destination). Lots of mentions of smbd & NT_STATUS_DISK_FULL The boot USB has 745MB used, so that's not full Memory is being heavily used but it's cached memory rather than processes so that's not actually 'full' Dockers are still mostly up - they just struggle to write to anything Disks are all detected, accessible and have 10's of GB of free space (50GB+ on all disks with ~87% usage on the entire array) I've attached some files with various outputs but I can't get diagnostics to run or save so that's a no-go unfortunately ๐Ÿ˜” diagnostics.txt free-m.txt syslog.txt diagnostics.zip
  3. By that point in the logs (the first occurrence of csrf token), unraid had already caught fire, so to speak. I had a couple of tabs open on the same machine after seeing all of the issues. Closing all of them and starting afresh stopped any further csrf warnings but didn't predate the 'undetermined' state of the array, nor the vanishing disks on the dashboard/main tabs. I'll have to wait for it to happen again, unfortunately ๐Ÿ˜ญ I rebooted so that I could be sure all of my overnight actions would run successfully (appdata & usb backup, mover, trim, etc.) Completely forgot about manual diagnostics via SSH ๐Ÿคฆโ€โ™€๏ธ it's been a long while since anything has gone horribly wrong ๐Ÿ˜… which is a silver lining I suppose!
  4. Dockers seem to be working, I can access SMB shares but disks vanish from the Dashboard & Main tab and browsing files through the Unraid UI brings up errors but appears to mostly work (I think) Warning: syntax error, unexpected end of file, expecting TC_DOLLAR_CURLY or TC_QUOTED_STRING or '"' in state/disks.ini on line 240 in /usr/local/emhttp/plugins/dynamix/template.php on line 36 Warning: syntax error, unexpected end of file, expecting TC_DOLLAR_CURLY or TC_QUOTED_STRING or '"' in state/disks.ini on line 240 in /usr/local/emhttp/plugins/dynamix/include/Browse.php on line 44 Sadly, I'm unable to get a copy of the syslog or diagnostics โ€“ they both 404 ๐Ÿ˜” And here's what the dashboard looks like: Sep 28 01:16:49 Raptor CA Backup/Restore: ####################### Sep 28 01:16:49 Raptor CA Backup/Restore: appData Backup complete Sep 28 01:16:49 Raptor CA Backup/Restore: ####################### Sep 28 01:16:49 Raptor CA Backup/Restore: Deleting /mnt/user/Backups/unRAID/Appdata/[email protected] Sep 28 01:16:49 Raptor avahi-daemon[3790]: Joining mDNS multicast group on interface vethb85f084.IPv6 with address fe80::8c65:47ff:fe95:b3f. Sep 28 01:16:49 Raptor avahi-daemon[3790]: New relevant interface vethb85f084.IPv6 for mDNS. Sep 28 01:16:49 Raptor avahi-daemon[3790]: Registering new address record for fe80::8c65:47ff:fe95:b3f on vethb85f084.*. Sep 28 01:16:49 Raptor avahi-daemon[3790]: Joining mDNS multicast group on interface veth2debf98.IPv6 with address fe80::e87a:86ff:fe47:5f0b. Sep 28 01:16:49 Raptor avahi-daemon[3790]: New relevant interface veth2debf98.IPv6 for mDNS. Sep 28 01:16:49 Raptor avahi-daemon[3790]: Registering new address record for fe80::e87a:86ff:fe47:5f0b on veth2debf98.*. Sep 28 01:16:49 Raptor avahi-daemon[3790]: Joining mDNS multicast group on interface veth715e3c6.IPv6 with address fe80::f441:50ff:fe0d:8aec. Sep 28 01:16:49 Raptor avahi-daemon[3790]: New relevant interface veth715e3c6.IPv6 for mDNS. Sep 28 01:16:49 Raptor avahi-daemon[3790]: Registering new address record for fe80::f441:50ff:fe0d:8aec on veth715e3c6.*. Sep 28 01:16:50 Raptor avahi-daemon[3790]: Joining mDNS multicast group on interface veth0c8a49f.IPv6 with address fe80::30ce:daff:fe4e:4dcd. Sep 28 01:16:50 Raptor avahi-daemon[3790]: New relevant interface veth0c8a49f.IPv6 for mDNS. Sep 28 01:16:50 Raptor avahi-daemon[3790]: Registering new address record for fe80::30ce:daff:fe4e:4dcd on veth0c8a49f.*. Sep 28 01:16:50 Raptor avahi-daemon[3790]: Joining mDNS multicast group on interface veth9de59c5.IPv6 with address fe80::fc99:8dff:fe8e:aa1d. Sep 28 01:16:50 Raptor avahi-daemon[3790]: New relevant interface veth9de59c5.IPv6 for mDNS. Sep 28 01:16:50 Raptor avahi-daemon[3790]: Registering new address record for fe80::fc99:8dff:fe8e:aa1d on veth9de59c5.*. Sep 28 01:17:01 Raptor CA Backup/Restore: Backup / Restore Completed Sep 28 01:17:01 Raptor emhttpd: read SMART /dev/sdb Sep 28 01:17:01 Raptor emhttpd: read SMART /dev/sdc Sep 28 01:17:02 Raptor emhttpd: read SMART /dev/sdk Sep 28 01:17:02 Raptor emhttpd: read SMART /dev/sdi Sep 28 01:17:09 Raptor emhttpd: read SMART /dev/sdh Sep 28 01:17:16 Raptor emhttpd: read SMART /dev/sdg Sep 28 01:17:24 Raptor emhttpd: read SMART /dev/sdl Sep 28 01:17:35 Raptor emhttpd: read SMART /dev/sde Sep 28 01:17:47 Raptor emhttpd: read SMART /dev/sdd Sep 28 01:46:46 Raptor emhttpd: spinning down /dev/sdj Sep 28 01:47:03 Raptor emhttpd: spinning down /dev/sdk Sep 28 01:47:50 Raptor emhttpd: spinning down /dev/sdd Sep 28 01:47:50 Raptor emhttpd: spinning down /dev/sdc Sep 28 01:47:53 Raptor emhttpd: spinning down /dev/sdi Sep 28 01:47:55 Raptor emhttpd: spinning down /dev/sdh Sep 28 01:47:56 Raptor emhttpd: spinning down /dev/sdg Sep 28 01:47:56 Raptor emhttpd: spinning down /dev/sdf Sep 28 01:47:56 Raptor emhttpd: spinning down /dev/sdl Sep 28 01:48:18 Raptor emhttpd: spinning down /dev/sde Sep 28 01:48:18 Raptor emhttpd: spinning down /dev/sdb Sep 28 02:04:18 Raptor emhttpd: read SMART /dev/sdl Sep 28 02:04:30 Raptor emhttpd: read SMART /dev/sdb Sep 28 02:04:42 Raptor emhttpd: read SMART /dev/sdd Sep 28 02:04:48 Raptor emhttpd: read SMART /dev/sdh Sep 28 02:04:56 Raptor emhttpd: read SMART /dev/sdc Sep 28 02:05:03 Raptor emhttpd: read SMART /dev/sdf Sep 28 02:05:16 Raptor emhttpd: read SMART /dev/sdg Sep 28 02:05:24 Raptor emhttpd: read SMART /dev/sdi Sep 28 02:56:49 Raptor emhttpd: spinning down /dev/sdc Sep 28 02:56:59 Raptor emhttpd: spinning down /dev/sdh Sep 28 02:56:59 Raptor emhttpd: spinning down /dev/sdg Sep 28 02:57:00 Raptor emhttpd: spinning down /dev/sdb Sep 28 02:57:00 Raptor emhttpd: spinning down /dev/sdl Sep 28 02:57:00 Raptor emhttpd: spinning down /dev/sdi Sep 28 02:57:02 Raptor emhttpd: spinning down /dev/sdd Sep 28 02:57:49 Raptor emhttpd: spinning down /dev/sdf Sep 28 03:39:12 Raptor emhttpd: read SMART /dev/sdl Sep 28 03:40:12 Raptor emhttpd: read SMART /dev/sdb Sep 28 03:40:29 Raptor emhttpd: read SMART /dev/sdh Sep 28 03:40:37 Raptor emhttpd: read SMART /dev/sdc Sep 28 03:40:46 Raptor emhttpd: read SMART /dev/sdi Sep 28 03:41:19 Raptor emhttpd: read SMART /dev/sdg Sep 28 03:41:41 Raptor emhttpd: read SMART /dev/sdf Sep 28 04:11:06 Raptor emhttpd: spinning down /dev/sdc Sep 28 04:11:11 Raptor emhttpd: spinning down /dev/sdl Sep 28 04:11:13 Raptor emhttpd: spinning down /dev/sdi Sep 28 04:11:33 Raptor emhttpd: spinning down /dev/sdg Sep 28 04:11:36 Raptor emhttpd: spinning down /dev/sdh Sep 28 04:11:58 Raptor emhttpd: spinning down /dev/sdf Sep 28 04:12:00 Raptor emhttpd: spinning down /dev/sdb Sep 28 08:00:01 Raptor emhttpd: read SMART /dev/sdk Sep 28 08:00:02 Raptor emhttpd: read SMART /dev/sdj Sep 28 08:00:20 Raptor root: /mnt/staging: 5.7 GiB (6158344192 bytes) trimmed on /dev/sdk1 Sep 28 08:00:20 Raptor root: /mnt/scratch: 11.1 GiB (11889082368 bytes) trimmed on /dev/sdj1 Sep 28 08:00:20 Raptor root: /mnt/cache: 8 GiB (8593104896 bytes) trimmed on /dev/nvme0n1p1 Sep 28 08:27:02 Raptor emhttpd: read SMART /dev/sdf Sep 28 08:27:14 Raptor emhttpd: read SMART /dev/sdb Sep 28 08:30:03 Raptor emhttpd: spinning down /dev/sdj Sep 28 08:30:03 Raptor emhttpd: spinning down /dev/sdk Sep 28 08:57:15 Raptor emhttpd: spinning down /dev/sdb Sep 28 08:57:27 Raptor emhttpd: spinning down /dev/sdf Sep 28 08:59:02 Raptor kernel: vethff49b5f: renamed from eth0 Sep 28 09:29:01 Raptor emhttpd: read SMART /dev/sdb Sep 28 09:29:13 Raptor emhttpd: read SMART /dev/sdh Sep 28 09:29:23 Raptor emhttpd: read SMART /dev/sdi Sep 28 09:29:31 Raptor emhttpd: read SMART /dev/sdd Sep 28 09:29:31 Raptor emhttpd: read SMART /dev/sdc Sep 28 09:29:46 Raptor emhttpd: read SMART /dev/sdg Sep 28 09:30:08 Raptor emhttpd: read SMART /dev/sdf Sep 28 09:30:27 Raptor emhttpd: read SMART /dev/sdl Sep 28 10:00:56 Raptor emhttpd: spinning down /dev/sdh Sep 28 10:01:04 Raptor emhttpd: spinning down /dev/sdf Sep 28 10:01:04 Raptor emhttpd: spinning down /dev/sdl Sep 28 10:01:06 Raptor emhttpd: spinning down /dev/sdg Sep 28 10:01:06 Raptor emhttpd: spinning down /dev/sdd Sep 28 10:01:06 Raptor emhttpd: spinning down /dev/sdb Sep 28 10:01:06 Raptor emhttpd: spinning down /dev/sdi Sep 28 10:01:10 Raptor emhttpd: spinning down /dev/sdc Sep 28 11:38:24 Raptor kernel: eth0: renamed from vethc7d238e Sep 28 12:14:00 Raptor kernel: docker0: port 2(veth3105c7e) entered disabled state Sep 28 12:14:00 Raptor kernel: vethf8bf137: renamed from eth0 Sep 28 12:14:00 Raptor avahi-daemon[3790]: Interface veth3105c7e.IPv6 no longer relevant for mDNS. Sep 28 12:14:00 Raptor avahi-daemon[3790]: Leaving mDNS multicast group on interface veth3105c7e.IPv6 with address fe80::1812:b1ff:fefc:2a1e. Sep 28 12:14:00 Raptor kernel: docker0: port 2(veth3105c7e) entered disabled state Sep 28 12:14:00 Raptor kernel: device veth3105c7e left promiscuous mode Sep 28 12:14:00 Raptor kernel: docker0: port 2(veth3105c7e) entered disabled state Sep 28 12:14:00 Raptor avahi-daemon[3790]: Withdrawing address record for fe80::1812:b1ff:fefc:2a1e on veth3105c7e. Sep 28 12:14:00 Raptor kernel: docker0: port 2(veth387de0f) entered blocking state Sep 28 12:14:00 Raptor kernel: docker0: port 2(veth387de0f) entered disabled state Sep 28 12:14:00 Raptor kernel: device veth387de0f entered promiscuous mode Sep 28 12:14:00 Raptor kernel: docker0: port 2(veth387de0f) entered blocking state Sep 28 12:14:00 Raptor kernel: docker0: port 2(veth387de0f) entered forwarding state Sep 28 12:14:00 Raptor kernel: eth0: renamed from vethc4c3331 Sep 28 12:14:00 Raptor kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth387de0f: link becomes ready Sep 28 12:14:02 Raptor avahi-daemon[3790]: Joining mDNS multicast group on interface veth387de0f.IPv6 with address fe80::6c52:b2ff:fe8a:632b. Sep 28 12:14:02 Raptor avahi-daemon[3790]: New relevant interface veth387de0f.IPv6 for mDNS. Sep 28 12:14:02 Raptor avahi-daemon[3790]: Registering new address record for fe80::6c52:b2ff:fe8a:632b on veth387de0f.*. Sep 28 12:14:05 Raptor kernel: docker0: port 2(veth387de0f) entered disabled state Sep 28 12:14:05 Raptor kernel: vethc4c3331: renamed from eth0 Sep 28 12:14:05 Raptor avahi-daemon[3790]: Interface veth387de0f.IPv6 no longer relevant for mDNS. Sep 28 12:14:05 Raptor avahi-daemon[3790]: Leaving mDNS multicast group on interface veth387de0f.IPv6 with address fe80::6c52:b2ff:fe8a:632b. Sep 28 12:14:05 Raptor kernel: docker0: port 2(veth387de0f) entered disabled state Sep 28 12:14:05 Raptor kernel: device veth387de0f left promiscuous mode Sep 28 12:14:05 Raptor kernel: docker0: port 2(veth387de0f) entered disabled state Sep 28 12:14:06 Raptor avahi-daemon[3790]: Withdrawing address record for fe80::6c52:b2ff:fe8a:632b on veth387de0f. Sep 28 12:14:06 Raptor kernel: docker0: port 2(veth0a58b51) entered blocking state Sep 28 12:14:06 Raptor kernel: docker0: port 2(veth0a58b51) entered disabled state Sep 28 12:14:06 Raptor kernel: device veth0a58b51 entered promiscuous mode Sep 28 12:14:06 Raptor kernel: docker0: port 2(veth0a58b51) entered blocking state Sep 28 12:14:06 Raptor kernel: docker0: port 2(veth0a58b51) entered forwarding state Sep 28 12:14:06 Raptor kernel: eth0: renamed from veth1e8f41d Sep 28 12:14:06 Raptor kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth0a58b51: link becomes ready Sep 28 12:14:08 Raptor avahi-daemon[3790]: Joining mDNS multicast group on interface veth0a58b51.IPv6 with address fe80::c0f7:85ff:fea7:7ab1. Sep 28 12:14:08 Raptor avahi-daemon[3790]: New relevant interface veth0a58b51.IPv6 for mDNS. Sep 28 12:14:08 Raptor avahi-daemon[3790]: Registering new address record for fe80::c0f7:85ff:fea7:7ab1 on veth0a58b51.*. Sep 28 13:06:02 Raptor emhttpd: read SMART /dev/sdl Sep 28 13:16:53 Raptor emhttpd: read SMART /dev/sdf Sep 28 13:17:05 Raptor emhttpd: read SMART /dev/sdb Sep 28 13:17:18 Raptor emhttpd: read SMART /dev/sdg Sep 28 13:17:25 Raptor emhttpd: read SMART /dev/sdh Sep 28 13:17:34 Raptor emhttpd: read SMART /dev/sdi Sep 28 13:18:13 Raptor emhttpd: read SMART /dev/sdc Sep 28 13:47:18 Raptor emhttpd: spinning down /dev/sdb Sep 28 13:47:36 Raptor emhttpd: spinning down /dev/sdh Sep 28 13:47:36 Raptor emhttpd: spinning down /dev/sdi Sep 28 13:47:43 Raptor emhttpd: spinning down /dev/sdg Sep 28 13:48:14 Raptor emhttpd: spinning down /dev/sdc Sep 28 14:10:11 Raptor emhttpd: spinning down /dev/sdf Sep 28 14:32:26 Raptor emhttpd: read SMART /dev/sdk Sep 28 15:02:46 Raptor emhttpd: spinning down /dev/sdk Sep 28 16:22:20 Raptor emhttpd: read SMART /dev/sdb Sep 28 16:26:12 Raptor nginx: 2022/09/28 16:26:12 [crit] 16469#16469: *2270300 pwritev() "/var/lib/nginx/client_body/0000223547" failed (28: No space left on device), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:26:27 Raptor nginx: 2022/09/28 16:26:27 [crit] 16469#16469: *2270445 pwritev() "/var/lib/nginx/client_body/0000223555" has written only 4096 of 9068, client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:27:01 Raptor nginx: 2022/09/28 16:27:01 [crit] 16469#16469: *2270781 pwrite() "/var/lib/nginx/client_body/0000223571" failed (28: No space left on device), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:27:22 Raptor nginx: 2022/09/28 16:27:22 [crit] 16469#16469: *2270944 pwritev() "/var/lib/nginx/client_body/0000223572" has written only 4096 of 9068, client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:27:33 Raptor nginx: 2022/09/28 16:27:33 [crit] 16469#16469: *2271044 pwritev() "/var/lib/nginx/client_body/0000223573" failed (28: No space left on device), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:30:09 Raptor nginx: 2022/09/28 16:30:09 [crit] 16469#16469: *2272423 pwritev() "/var/lib/nginx/client_body/0000223574" has written only 4096 of 9068, client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:30:11 Raptor unraid-api[7523]: โš ๏ธ Caught exception: Maximum call stack size exceeded Sep 28 16:30:11 Raptor unraid-api[7523]: โš ๏ธ UNRAID API crashed with exit code 1 Sep 28 16:30:51 Raptor nginx: 2022/09/28 16:30:51 [crit] 16469#16469: *2272850 pwritev() "/var/lib/nginx/client_body/0000223575" has written only 4096 of 9068, client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:30:52 Raptor nginx: 2022/09/28 16:30:52 [crit] 16469#16469: *2272855 pwritev() "/var/lib/nginx/client_body/0000223576" failed (28: No space left on device), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:31:01 Raptor winbindd[3762]: [2022/09/28 16:31:01.482777, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:31:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:32:01 Raptor winbindd[3762]: [2022/09/28 16:32:01.565444, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:32:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:33:01 Raptor winbindd[3762]: [2022/09/28 16:33:01.646888, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:33:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:34:01 Raptor winbindd[3762]: [2022/09/28 16:34:01.703275, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:34:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:34:19 Raptor nginx: 2022/09/28 16:34:19 [crit] 16469#16469: *2275193 pwritev() "/var/lib/nginx/client_body/0000223577" failed (28: No space left on device), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:35:01 Raptor winbindd[3762]: [2022/09/28 16:35:01.785317, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:35:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:36:01 Raptor winbindd[3762]: [2022/09/28 16:36:01.872562, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:36:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:36:56 Raptor winbindd[3762]: [2022/09/28 16:36:56.363827, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:36:56 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:37:01 Raptor winbindd[3762]: [2022/09/28 16:37:01.948671, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:37:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:37:13 Raptor nginx: 2022/09/28 16:37:13 [crit] 16469#16469: *2277199 pwritev() "/var/lib/nginx/client_body/0000223578" has written only 8192 of 9068, client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:38:01 Raptor winbindd[3762]: [2022/09/28 16:38:01.034477, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:38:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:38:14 Raptor nginx: 2022/09/28 16:38:14 [crit] 16469#16469: *2277811 pwritev() "/var/lib/nginx/client_body/0000223583" has written only 4096 of 9068, client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:38:16 Raptor nginx: 2022/09/28 16:38:16 [crit] 16469#16469: *2277832 pwrite() "/var/lib/nginx/client_body/0000223584" failed (28: No space left on device), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:38:38 Raptor nginx: 2022/09/28 16:38:38 [crit] 16469#16469: *2277999 pwritev() "/var/lib/nginx/client_body/0000223585" has written only 4096 of 9068, client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:39:01 Raptor winbindd[3762]: [2022/09/28 16:39:01.123008, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:39:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:40:01 Raptor winbindd[3762]: [2022/09/28 16:40:01.210119, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:40:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:40:02 Raptor nginx: 2022/09/28 16:40:02 [crit] 16469#16469: *2278742 pwritev() "/var/lib/nginx/client_body/0000223586" failed (28: No space left on device), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:40:18 Raptor nginx: 2022/09/28 16:40:18 [crit] 16469#16469: *2278886 pwritev() "/var/lib/nginx/client_body/0000223587" failed (28: No space left on device), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:41:01 Raptor winbindd[3762]: [2022/09/28 16:41:01.284950, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:41:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:41:14 Raptor nginx: 2022/09/28 16:41:14 [crit] 16469#16469: *2279381 pwritev() "/var/lib/nginx/client_body/0000223588" has written only 4096 of 9068, client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:42:01 Raptor winbindd[3762]: [2022/09/28 16:42:01.356158, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:42:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:43:01 Raptor winbindd[3762]: [2022/09/28 16:43:01.440045, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:43:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:44:01 Raptor winbindd[3762]: [2022/09/28 16:44:01.514730, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:44:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:45:01 Raptor winbindd[3762]: [2022/09/28 16:45:01.599798, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:45:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:46:01 Raptor winbindd[3762]: [2022/09/28 16:46:01.687302, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:46:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:46:19 Raptor nginx: 2022/09/28 16:46:19 [crit] 16469#16469: *2282696 pwritev() "/var/lib/nginx/client_body/0000223589" failed (28: No space left on device), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:46:29 Raptor root: error: /plugins/community.applications/scripts/notices.php: wrong csrf_token Sep 28 16:46:29 Raptor root: error: /webGui/include/Report.php: wrong csrf_token Sep 28 16:46:29 Raptor root: error: /webGui/include/FileTree.php: wrong csrf_token Sep 28 16:46:59 Raptor winbindd[3762]: [2022/09/28 16:46:59.387775, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:46:59 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:47:00 Raptor unraid-api[22543]: โœ”๏ธ UNRAID API started successfully! Sep 28 16:47:01 Raptor winbindd[3762]: [2022/09/28 16:47:01.771620, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:47:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:47:01 Raptor winbindd[3762]: [2022/09/28 16:47:01.794217, 0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn) Sep 28 16:47:01 Raptor winbindd[3762]: open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED Sep 28 16:47:01 Raptor nginx: 2022/09/28 16:47:01 [crit] 16469#16469: *2283241 pwritev() "/var/lib/nginx/client_body/0000223620" failed (28: No space left on device), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:47:10 Raptor nginx: 2022/09/28 16:47:10 [crit] 16469#16469: *2283344 pwritev() "/var/lib/nginx/client_body/0000223625" has written only 4096 of 9068, client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost" Sep 28 16:47:12 Raptor nginx: 2022/09/28 16:47:12 [crit] 16469#16469: *2283370 pwrite() "/var/lib/nginx/client_body/0000223626" failed (28: No space left on device), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost"
  5. I disagree ๐Ÿ˜…๐Ÿ˜… the spinup mechanic has changed between b35 & rc1, resulting in an undesired effect - Telegraf monitoring smart temps lets me pull temperatures & smart data into Grafana - previously if disks were spun down they would not be spun up by my telegraf.conf - the only variable here was the new Unraid version ๐Ÿ‘€ now disks are spun up if a smart command grazes them ๐Ÿค” (either that or the disks are no longer reporting their standby status correctly/in the same way) # ## Skip checking disks in this power mode. Defaults to # ## "standby" to not wake up disks that have stopped rotating. # ## See --nocheck in the man pages for smartctl. # ## smartctl version 5.41 and 5.42 have faulty detection of # ## power mode and might require changing this value to # ## "never" depending on your disks. # nocheck = "standby"
  6. SMART checks are the trigger here, I've narrowed it down to my Telegraf container, the disks all spin up on RC1 where they didn't on Beta35 (I've tried with & without `nocheck= "standby"` commented out) Edit: I should also add that if I comment out `[[inputs.smart]]` the disks don't spin up while this container is active.
  7. Seems I'm seeing a lack of spindowns too on RC1 ๐Ÿฅด I've attached a diagnostic with all devices spun up ... then after I've manually spun the array down and all the devices have spun back up again 'on their own' (possibly due to SMART).raptor-diagnostics-20201211-1819.zipraptor-diagnostics-20201211-1814.zip
  8. Apologies, I'm on the same version as you, I only updated to it on July 1st but I've not been getting along with it haha ๐Ÿ˜…
  9. Jul 2 22:24:16 Raptor root: plugin: running: anonymous Jul 2 22:27:06 Raptor emhttpd: cmd: /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin install https://raw.githubusercontent.com/bergware/dynamix/a088b81081160482030019ccd61790428bbb805c/unRAIDv6/dynamix.system.autofan.plg Jul 2 22:27:13 Raptor emhttpd: cmd: /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin remove dynamix.system.autofan.plg Jul 2 22:27:13 Raptor root: plugin: running: anonymous Jul 2 22:27:18 Raptor emhttpd: cmd: /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin install https://raw.githubusercontent.com/bergware/dynamix/a088b81081160482030019ccd61790428bbb805c/unRAIDv6/dynamix.system.autofan.plg Jul 2 22:27:18 Raptor root: plugin: running: anonymous Jul 2 22:27:18 Raptor root: plugin: creating: /boot/config/plugins/dynamix.system.autofan/dynamix.system.autofan.txz - downloading from URL https://raw.githubusercontent.com/bergware/dynamix/master/archive/dynamix.system.autofan.txz Jul 2 22:27:18 Raptor root: plugin: checking: /boot/config/plugins/dynamix.system.autofan/dynamix.system.autofan.txz - MD5 Jul 2 22:27:38 Raptor emhttpd: cmd: /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin install https://raw.githubusercontent.com/bergware/dynamix/a088b81081160482030019ccd61790428bbb805c/unRAIDv6/dynamix.system.autofan.plg Jul 2 22:27:38 Raptor root: plugin: running: anonymous Jul 2 22:27:38 Raptor root: plugin: creating: /boot/config/plugins/dynamix.system.autofan/dynamix.system.autofan.txz - downloading from URL https://raw.githubusercontent.com/bergware/dynamix/master/archive/dynamix.system.autofan.txz Jul 2 22:27:39 Raptor root: plugin: checking: /boot/config/plugins/dynamix.system.autofan/dynamix.system.autofan.txz - MD5 Jul 2 22:29:26 Raptor login[15500]: ROOT LOGIN on '/dev/pts/0' Jul 2 22:30:48 Raptor emhttpd: cmd: /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin install /boot/config/plugins/dynamix.system.autofan.plg Jul 2 22:30:48 Raptor root: plugin: running: anonymous Jul 2 22:30:48 Raptor root: plugin: creating: /boot/config/plugins/dynamix.system.autofan/dynamix.system.autofan.txz - downloading from URL https://raw.githubusercontent.com/bergware/dynamix/master/archive/dynamix.system.autofan.txz Jul 2 22:30:48 Raptor root: plugin: checking: /boot/config/plugins/dynamix.system.autofan/dynamix.system.autofan.txz - MD5 Jul 2 22:32:35 Raptor emhttpd: cmd: /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin install /boot/config/plugins/dynamix.system.autofan.plg Jul 2 22:32:35 Raptor root: plugin: running: anonymous Jul 2 22:32:35 Raptor root: plugin: creating: /boot/config/plugins/dynamix.system.autofan/dynamix.system.autofan.txz - downloading from URL https://raw.githubusercontent.com/bergware/dynamix/master/archive/dynamix.system.autofan.txz Jul 2 22:32:35 Raptor root: plugin: checking: /boot/config/plugins/dynamix.system.autofan/dynamix.system.autofan.txz - MD5 Jul 2 22:34:02 Raptor emhttpd: cmd: /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin_rm /boot/config/plugins-error/dynamix.system.autofan.plg Jul 2 22:34:22 Raptor emhttpd: cmd: /usr/local/emhttp/plugins/community.applications/scripts/pluginInstall.sh install https://raw.github.com/bergware/dynamix/master/unRAIDv6/dynamix.system.autofan.plg Jul 2 22:34:23 Raptor root: plugin: running: anonymous Jul 2 22:34:23 Raptor root: plugin: creating: /boot/config/plugins/dynamix.system.autofan/dynamix.system.autofan.txz - downloading from URL https://raw.githubusercontent.com/bergware/dynamix/master/archive/dynamix.system.autofan.txz Jul 2 22:34:23 Raptor root: plugin: checking: /boot/config/plugins/dynamix.system.autofan/dynamix.system.autofan.txz - MD5 Jul 2 22:34:23 Raptor root: plugin: running: /boot/config/plugins/dynamix.system.autofan/dynamix.system.autofan.txz Jul 2 22:34:23 Raptor root: plugin: creating: /tmp/start_service - from INLINE content Jul 2 22:34:23 Raptor root: plugin: setting: /tmp/start_service - mode to 0770 Jul 2 22:34:23 Raptor root: plugin: running: anonymous Jul 2 22:34:54 Raptor ool www[16948]: /usr/local/emhttp/plugins/dynamix.system.autofan/scripts/rc.autofan 'start' Jul 2 22:35:02 Raptor ool www[16948]: /usr/local/emhttp/plugins/dynamix.system.autofan/scripts/rc.autofan 'start' Jul 2 22:35:22 Raptor ool www[16061]: /usr/local/emhttp/plugins/dynamix.system.autofan/scripts/rc.autofan 'start' Jul 2 22:35:28 Raptor ool www[16948]: /usr/local/emhttp/plugins/dynamix.system.autofan/scripts/rc.autofan 'start' Jul 2 22:35:28 Raptor autofan: autofan process ID 17353 started, To terminate it, type: autofan -q -c /sys/class/hwmon/hwmon2/pwm1 -f /sys/class/hwmon/hwmon2/fan1_input Jul 2 22:35:33 Raptor autofan: Highest disk temp is 31C, adjusting fan speed from: FULL (100% @ 0rpm) to: OFF (0% @ 0rpm) Jul 2 22:35:41 Raptor ool www[17099]: /usr/local/emhttp/plugins/dynamix.system.autofan/scripts/rc.autofan 'start' Jul 2 22:35:54 Raptor ool www[17289]: /usr/local/emhttp/plugins/dynamix.system.autofan/scripts/rc.autofan 'start' Jul 2 22:36:41 Raptor ool www[17099]: /usr/local/emhttp/plugins/dynamix.system.autofan/scripts/rc.autofan 'start' Jul 2 22:37:01 Raptor ool www[17289]: /usr/local/emhttp/plugins/dynamix.system.autofan/scripts/rc.autofan 'start' Jul 2 22:37:01 Raptor autofan: autofan process ID 18017 started, To terminate it, type: autofan -q -c /sys/devices/platform/nct6775.656/hwmon/hwmon2/pwm2 -f /sys/devices/platform/nct6775.656/hwmon/hwmon2/fan4_input Jul 2 22:37:01 Raptor autofan: autofan excluding drives sdd,sde,sdf,sdi,sdj,sdk,sdl,sdg,sdh from max temp calculations Jul 2 22:37:06 Raptor autofan: Highest disk temp is 0C, adjusting fan speed from: FULL (100% @ 1400rpm) to: OFF (0% @ 508rpm) Error log doesn't seem to appear in the folder either ๐Ÿ‘€ root@Raptor:/boot/config/plugins-error# ls -lah total 16K drwx------ 2 root root 8.0K Jul 2 22:34 ./ drwx------ 8 root root 8.0K Jul 2 22:30 ../ root@Raptor:/boot/config/plugins-error# Can't get autofan to run against PWM 2 - it tries to save but never completes, defaults to PWM 1 and setting the plugin to "Disabled" doesn't disable it haha! Let me know if I can do anything to help out! - I tried to install an old plugin version from GitHub but it doesn't think the MD5 matches but I know the issue didn't exist in older versions of the plugin
  10. https://s3.amazonaws.com/dnld.lime-technology.com/next/unRAIDServer-6.9.0-beta1-x86_64.zip
  11. All sorted @Squid ๐Ÿ˜Œ Rebuild just finished and all is good in the world ๐Ÿ™Œ Thanks again!
  12. Thank you so much Squid!! Doing that now ๐Ÿ™๐Ÿ™๐Ÿ™
  13. Howdy folks! ๐Ÿ‘‹ The last time I had a disk in a similar state, I rushed things, screwed up and lost 3TB of media... Do'h ๐Ÿคฆโ€โ™‚๏ธ So I'd love to get a lovely expert's opinion on what to do next ๐Ÿ˜ Disk 3 is still open-able, no errors in the SMART data ๐Ÿ‘Œ What I've done so far: - Went to move some recent files which would've been moved from Cache to the Array within the last few days ๐Ÿ˜ƒ - I got a couple of emails from my NAS warning me that /dev/sdg had gone bad ๐Ÿ˜ฌ - I headed to the main page & saw the lovely โŒ - Checked the SMART data on the disk, seemed fine - Rebooted... I got another email to say "Subject: Notice [RAPTOR] - array turned good" - Still got a lovely emulated disk - Grabbed a diag dump & headed straight here โค๏ธ What should I do next? - I believe disk 3 to be healthy - I don't have physical access at the moment ๐Ÿ˜” (I'm a couple hundred miles away) - I do have full web / SSH (VPN) access And of course, thank you in advance for being amazing!! raptor-diagnostics-20191010-1615.zip
  14. Exactly the same here Jan 23 04:17:25 Raptor kernel: BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 3761, flush 0, corrupt 0, gen 0 Jan 23 04:17:30 Raptor kernel: BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 3762, flush 0, corrupt 0, gen 0 Jan 23 04:17:30 Raptor kernel: BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 3763, flush 0, corrupt 0, gen 0 Jan 23 04:17:30 Raptor kernel: BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 3764, flush 0, corrupt 0, gen 0 Jan 23 04:17:30 Raptor kernel: BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 3765, flush 0, corrupt 0, gen 0 Jan 23 04:17:30 Raptor kernel: BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 3766, flush 0, corrupt 0, gen 0 Jan 23 04:17:30 Raptor kernel: BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 3767, flush 0, corrupt 0, gen 0 Jan 23 04:17:30 Raptor kernel: BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 3768, flush 0, corrupt 0, gen 0 Jan 23 04:17:30 Raptor kernel: BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 3769, flush 0, corrupt 0, gen 0 raptor-diagnostics-20190123-0839.zip
  15. Is it possible to figure out the cause? Last thing I want to do is encounter the same issue again 'soon' Looks like I'd best get backing up in the meantime then
  16. Hmm... 2:16 Raptor kernel: loop: Write error at byte offset 2392465408, length 4096. Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672784 Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 2, rd 0, flush 1, corrupt 0, gen 0 Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 4215803904, length 4096. Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 8233992 Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 3, rd 0, flush 1, corrupt 0, gen 0 Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392465408, length 4096. Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672784 Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 4, rd 0, flush 1, corrupt 0, gen 0 Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392473600, length 4096. Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672800 Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 5, rd 0, flush 1, corrupt 0, gen 0 Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392608768, length 4096. Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4673064 Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 6, rd 0, flush 1, corrupt 0, gen 0 Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392465408, length 4096. Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672784 Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 7, rd 0, flush 1, corrupt 0, gen 0 Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 4367192064, length 4096. Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 8529672 Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 8, rd 0, flush 1, corrupt 0, gen 0 Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392465408, length 4096. Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672784 Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 9, rd 0, flush 1, corrupt 0, gen 0 Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392469504, length 4096. Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672792 Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 10, rd 0, flush 1, corrupt 0, gen 0 Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392489984, length 4096. Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672832 Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 11, rd 0 Crap? ------- Nov 15 21:31:57 Raptor kernel: BTRFS error (device sdl1): unable to find ref byte nr 2331165265920 parent 0 root 5 owner 18608628 offset 4014080 Nov 15 21:31:57 Raptor kernel: BTRFS: error (device sdl1) in __btrfs_free_extent:6950: errno=-2 No such entry Nov 15 21:31:57 Raptor kernel: BTRFS info (device sdl1): forced readonly Nov 15 21:31:57 Raptor kernel: BTRFS: error (device sdl1) in btrfs_run_delayed_refs:3057: errno=-2 No such entry Nov 15 21:31:57 Raptor kernel: BTRFS error (device sdl1): pending csums is 3751936 /dev/sdl
  17. I went to fire up Plex and my server was offline... I took a look at my NAS and it was true, Plex was stopped. In fact all of my containers had stopped! All of my disks were fine but my log was full of errors. I'm attaching the diagnostics if anybody wants to take a look. Latest version of Unraid (6.6.5) No idea what the cause was, I've just rebooted now so hopefully all is good ๐Ÿคž raptor-diagnostics-20181115-2127.zip
  18. sorted Just needed a balance in the end Ty
  19. Also got this email message this morning from my Unraid:
  20. btrfs fi show Label: none uuid: b5b7dc19-7d32-47df-be29-d6dd41e675d5 Total devices 2 FS bytes used 44.19GiB devid 4 size 111.79GiB used 69.09GiB path /dev/sdl1 *** Some devices missing My CACHE disk stays 'spun up' (SSD) but CACHE2 has no read / write activity and spins down when it hits the hour mark (the hour of disk inactivity). That command above shows device missing. CACHE is /dev/sdl & CACHE2 is /dev/sdk Both disks are there and showing without error... However the CACHE isn't treating them equally - How might I go about fixing this? Diagnostics attached. Thanks raptor-diagnostics-20181003-0836.zip
  21. Turns out it was two DOA WD's, after I ripped them out of the array and started prodding at them some more via Preclear & even Windows they started to show signs of badness! ๐Ÿ˜‚ WD agreed and are shipping some brand new drives out to replace these, yay... Guess I can't close a thread here But 'problem solved' ... I'm incredibly unlucky and now I've got an LSI HBA card, win win I suppose ๐Ÿคทโ€โ™‚๏ธ
  22. @johnnie.black Hey LSI card arrived, installed, flashed to IT mode, everything detected again... However I still get errors on Disk6 as soon as I stick it into the array ๐Ÿค” It passed a single preclear run with no errors but it hates being in the array - I'm doing another preclear but wanted to start my array and make it safe! So... I've set "No device" for disk6 - the contents will be emulated and thus if I experienced a REAL disk failure whilst in that state - I'd be screwed (right?) I had no data on disk6 so I'm not fussed about emulating an empty 3TB drive Am I correct in thinking I can take a screenshot now of the assignments, go to tools > New config and 'restart' with parity + disk1-5 and be in the green again? And let this preclear run in the background before I think about adding disk6 again? Cheers!!
  23. Oh, I was on the right lines? Excellent, I shall do so (yay more money to spend ๐Ÿ˜…) Cheers Johnnie! Fingers crossed! (Why me ๐Ÿ˜‚๐Ÿ˜ญ)
  24. Considering buying this: https://smile.amazon.co.uk/10Gtek-Internal-Express-Controller-SAS2008/dp/B01M2AC40Y/ref=sr_1_1?ie=UTF8&qid=1537007716&sr=8-1&keywords=SAS+controller+HBA And flashing it to IT mode: ftp://ftp.supermicro.nl/driver/SAS/LSI/2008/IR_IT/Firmware/IT/ (Use supermicro's FTP site to flash the LSI 3008 to IT mode all the time at work) ------ I've got 4hrs left to get this on Prime tomorrow (Sunday 16th) along with a SAS to SATA breakout cable... I know the disks are fine, they're brand new and underwent a preclear. It's not impossible for it to have failed, sure... But I'm strongly leaning towards the cheapo SATA card (https://www.scan.co.uk/products/lycom-pe-120-ahci-sata-iii-6gbps-internal-4-port-low-profile-pcie-20-host-adapter) Based on this; the 3 disks connected to it had 'errors' on the Unraid Main tab the second that I began to `cp` from disk5 to disk6 (errors appeared on disk4/5/6) and now here I am, a vanished disk4 & a disabled disk6! I don't want to start mixing drives around... I have screenshots of the assignments so that's ok. There's no spare ports, that card has been running with 1 disk on it for a over a year - seems that the moment I add a 2nd and then 3rd drive that it's showing 'weakness' great... I only have 6 onboard SATA + the 4 from this card. I've got no other HBAs or SATA cards, etc. etc. Hence why I'm considering the LSI card so I can hook all the HDDs up to a (hopefully much more solid) card and get the array started again...