paloooz

One of my drives was at 7 reallocated sectors for about a year. I acknowledged the error and kept using the drive as normal. Within the last month, the error count grew to 132 and then 164. At that point, I stopped all write operations to the array and began migrating data out of that drive using unbalance. During the migration, the error count grew once more to 1440. There is 1TB of data left to move, and the parity is protecting me. I'm going to replace this drive. In another server, my parity drive now has 24 reallocated sectors. This one is a bit trickier to replace because my data will not be redundant while parity rebuilds. I will report back here in the future with what I experienced as the drives failed. I have never seen a drive fail before, so I want to take these ones to the end.

Made a new unraid flash drive from backup and haven't seen the errors since. 🤷‍♂️

I started getting a some of these messages on one of my servers. They have appeared both on unraid 6.9.2 and 6.10.0-rc4 I noticed identical timestamps for traps lsof faults to messages related to an SMB share I have mounted through unassigned devices plugin. I switched to NFS and saw the same thing, identical timestamps like these: Apr 9 09:12:26 server1 kernel: nfs: server server2 not responding, still trying Apr 9 09:13:57 server1 kernel: nfs: server server2 not responding, still trying Apr 9 09:25:16 server1 kernel: nfs: server server2 OK Apr 9 09:25:16 server1 kernel: nfs: server server2 OK Apr 9 09:25:16 server1 kernel: traps: lsof[28800] general protection fault ip:14789875068e sp:d00e994a5e9bf867 error:0 in libc-2.33.so[147898737000+15e000] The errors are not isolated to these remote share events however. I see them over night every few minutes sometimes. They go away when I disable all dockers that write data to the array. The dockers that I have set up writing data to the array are set up to skip cache and write directly to disk by making use of the user0 path. Dockers writing to cache directly appear not to cause these errors. I changed my RAM out this morning right before the above logs, so it does not appear to have been a RAM issue. To give you an idea of the frequency: Apr 9 13:01:21 bulk kernel: traps: lsof[18703] general protection fault ip:152691a7068e sp:67cb16bcd9202d51 error:0 in libc-2.33.so[152691a57000+15e000] Apr 9 13:03:40 bulk kernel: traps: lsof[29553] general protection fault ip:14ca3860268e sp:338366b21378c151 error:0 in libc-2.33.so[14ca385e9000+15e000] Apr 9 13:05:58 bulk kernel: traps: lsof[6002] general protection fault ip:15283c13568e sp:88599c663a7d2d0c error:0 in libc-2.33.so[15283c11c000+15e000] Apr 9 13:10:56 bulk flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup update Apr 9 13:12:55 bulk kernel: traps: lsof[10609] general protection fault ip:146da3be068e sp:60f3285e476ce07 error:0 in libc-2.33.so[146da3bc7000+15e000] Apr 9 13:17:16 bulk kernel: traps: lsof[29430] general protection fault ip:15253fe2c68e sp:2852dea9469dbc5a error:0 in libc-2.33.so[15253fe13000+15e000] Apr 9 13:20:44 bulk kernel: traps: lsof[10699] general protection fault ip:14d0d223568e sp:7d8d4dac2bae93db error:0 in libc-2.33.so[14d0d221c000+15e000] Apr 9 13:22:09 bulk kernel: traps: lsof[16076] general protection fault ip:1551bedfc68e sp:52eae7af2bc3ef6e error:0 in libc-2.33.so[1551bede3000+15e000] Apr 9 13:27:02 bulk kernel: traps: lsof[3421] general protection fault ip:147ada15e68e sp:94e24acb6feaf366 error:0 in libc-2.33.so[147ada145000+15e000] Apr 9 13:28:52 bulk kernel: traps: lsof[10829] general protection fault ip:14fab510468e sp:dcf1ce016ce7e31d error:0 in libc-2.33.so[14fab50eb000+15e000] Apr 9 13:33:44 bulk kernel: traps: lsof[866] general protection fault ip:14fea388368e sp:6202c7831c7058f5 error:0 in libc-2.33.so[14fea386a000+15e000] Apr 9 13:34:12 bulk kernel: traps: lsof[1970] general protection fault ip:15231c87968e sp:c564f62890ca273b error:0 in libc-2.33.so[15231c860000+15e000] Apr 9 13:36:01 bulk kernel: traps: lsof[9777] general protection fault ip:1538737e068e sp:1c7d7af3ca302571 error:0 in libc-2.33.so[1538737c7000+15e000] Apr 9 13:37:35 bulk kernel: traps: lsof[15971] general protection fault ip:146d7922b68e sp:3c798e2d5e1daad7 error:0 in libc-2.33.so[146d79212000+15e000] Apr 9 13:39:17 bulk kernel: traps: lsof[22890] general protection fault ip:14a33fccf68e sp:3d806913fb4d7a8a error:0 in libc-2.33.so[14a33fcb6000+15e000] Apr 9 13:40:14 bulk kernel: traps: lsof[24905] general protection fault ip:1496e82f468e sp:4c570071697aea07 error:0 in libc-2.33.so[1496e82db000+15e000] Apr 9 13:41:58 bulk kernel: traps: lsof[2591] general protection fault ip:146d1f16c68e sp:f31715d633cd8921 error:0 in libc-2.33.so[146d1f153000+15e000] Apr 9 13:43:01 bulk kernel: nfs: server BANG not responding, still trying Apr 9 13:43:08 bulk kernel: traps: lsof[7297] general protection fault ip:1541cf31d68e sp:c2cadcf58e5e9898 error:0 in libc-2.33.so[1541cf304000+15e000] Apr 9 13:44:14 bulk kernel: nfs: server BANG OK Apr 9 13:44:30 bulk kernel: traps: lsof[12570] general protection fault ip:146bb830e68e sp:649e70fe846dd957 error:0 in libc-2.33.so[146bb82f5000+15e000] Apr 9 13:47:06 bulk kernel: traps: lsof[23684] general protection fault ip:150b515cd68e sp:f2a66970a858088c error:0 in libc-2.33.so[150b515b4000+15e000] Apr 9 13:48:14 bulk kernel: traps: lsof[29120] general protection fault ip:1499e964268e sp:e727255ff67338f4 error:0 in libc-2.33.so[1499e9629000+15e000] Apr 9 13:50:54 bulk kernel: traps: lsof[4929] general protection fault ip:146b3cb0868e sp:3b533ce96fcf7b09 error:0 in libc-2.33.so[146b3caef000+15e000] Apr 9 13:52:01 bulk kernel: traps: lsof[12105] general protection fault ip:152b62cd668e sp:b4a9f0e886c059f error:0 in libc-2.33.so[152b62cbd000+15e000] Apr 9 13:53:31 bulk kernel: traps: lsof[18861] general protection fault ip:14dfbdae168e sp:240599e5c6592472 error:0 in libc-2.33.so[14dfbdac8000+15e000] Apr 9 13:54:31 bulk kernel: traps: lsof[24021] general protection fault ip:152fa3c9368e sp:dd9c15ccd5348c37 error:0 in libc-2.33.so[152fa3c7a000+15e000] Apr 9 14:03:59 bulk kernel: traps: lsof[30654] general protection fault ip:1532a722968e sp:80dca1ffff5b1d04 error:0 in libc-2.33.so[1532a7210000+15e000] Apr 9 14:10:28 bulk kernel: traps: lsof[3294] general protection fault ip:14fbf6ea468e sp:f1848198a3f43885 error:0 in libc-2.33.so[14fbf6e8b000+15e000] Apr 9 14:11:34 bulk kernel: traps: lsof[7685] general protection fault ip:14850b44c68e sp:fc326022fc09bce0 error:0 in libc-2.33.so[14850b433000+15e000] Apr 9 14:14:02 bulk kernel: traps: lsof[17615] general protection fault ip:154f6bed868e sp:f0799e54df766d9d error:0 in libc-2.33.so[154f6bebf000+15e000] Apr 9 14:16:53 bulk kernel: traps: lsof[31368] general protection fault ip:147390d1f68e sp:7ddb862d78afeddd error:0 in libc-2.33.so[147390d06000+15e000] Apr 9 14:17:14 bulk kernel: traps: lsof[1131] general protection fault ip:14b07b1b868e sp:4efd331921f54146 error:0 in libc-2.33.so[14b07b19f000+15e000] Going to try unplugging the USB keyboard I had plugged in per this reddit user's experience: https://www.reddit.com/r/unRAID/comments/ohakff/server_shutting_itself_down_at_least_once_a_day/ Update: Made no difference. Attempting changes to docker share access modes. (Again no difference)

This setup is now in the past for me. It lasted about 2 years before I was out of hard drive slots and needed to horizontally scale. The Antec 1200 case is great, and I am now going to look into selling it. I wonder if I can do that here on the unraid forums! I actually have TWO Antec 1200 cases, one is pictured above and has all of the hotswap bays in it already, the other has the normal faceplates. Anyway, here's what I'm running now in case you're wondering:

I just want to report my experience with Tdarr. I've been running Tdarr in multiple configurations with multiple nodes across two Unraid servers. Currently, I'm running two tdarr servers with four nodes operating on two independent data sets totaling around 90k files on 20 disks with parity. Temporary files are written directly to disk, skipping cache. I have never had any issues like this before, and I've saved almost 40TB over the last 5 months.

I recently built a new system using a used Antec Twelve Hundred I got on ebay. This case is in near perfect condition and the auction for it went up to around $450. I suspect a fellow unraid user had similar plans with it as I did. I decided to go with 3 x 2 hotswap bays for a total of 18 slots rather than the 4 in 3 or 5 x 3 caddies. Better air flow, but only 18 slots instead of 20. Close enough! The main reason is so I wouldn't have to modify the slots on the inside of the case to accommodate the flat sides of the 5 x 3 caddies. I'm attaching an image. 152TB usable storage now with room to expand. 4 empty slots, 10 x 12tb drives, 4 x 16tb drives (2 are parity).

Latest dashboard attached