October 2, 20178 yr My system is: ASRock C2750d4i Corsair 450W 16GB Crucial ECC RAM 512GB Crucial SSD (cache) 4x 4TB WD Reds 2x 8TB WD Red 1x 8TB WD Red (parity) It all started today. I installed nextcloud, and noticed that the port was closed. I thought if I reboot my server would fix the problem. After rebooting the server, I created a much bigger issue. The cache drive shows unmountable. All my docker images, appdata, and VMs are all on my cache drive. This is exactly what I did: 1. stopped all my docker containers 2. stopped all my VMs 3. stopped the array 4. rebooted 5. after the reboot, started the array 6. noticecd that cache drive is unmountable I did some more troubleshooting. I powered down the server and unplugged and replugged the cache disk cable, but no luck. It is still showing unmountable Edited October 15, 20178 yr by pingmanping add more information and attached the diagnostic
October 3, 20178 yr Author Here is the btrfsck /dev/sdc1 root@BGDunRAID:~# btrfsck /dev/sdc1 Checking filesystem on /dev/sdc1 UUID: b56a1610-bb12-400e-9b26-904a0e5bb338 checking extents bad key ordering 19 20 bad block 281542656 Errors found in extent allocation tree or chunk allocation checking free space cache checking fs roots bad key ordering 19 20 root 5 inode 13583 errors 400, nbytes wrong root 5 inode 1300023 errors 2001, no inode item, link count wrong unresolved ref dir 257 index 3 namelen 7 name libvirt filetype 2 errors 4, no inode ref root 5 inode 1300025 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 26 namelen 7 name domains filetype 2 errors 4, no inode ref root 5 inode 3765760 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 21 namelen 5 name unifi filetype 2 errors 4, no inode ref root 5 inode 3769246 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 22 namelen 10 name EmbyServer filetype 2 errors 4, no inode ref root 5 inode 3929077 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 25 namelen 9 name booksonic filetype 2 errors 4, no inode ref root 5 inode 4002362 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 34 namelen 16 name domains_no-cache filetype 2 errors 4, no inode ref root 5 inode 4009051 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 27 namelen 16 name binhex-delugevpn filetype 2 errors 4, no inode ref root 5 inode 4009160 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 28 namelen 6 name radarr filetype 2 errors 4, no inode ref root 5 inode 4009380 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 34 namelen 13 name binhex-sonarr filetype 2 errors 4, no inode ref root 5 inode 4135121 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 77 namelen 5 name Media filetype 2 errors 4, no inode ref root 5 inode 4374768 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 36 namelen 6 name apache filetype 2 errors 4, no inode ref root 5 inode 4378599 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 37 namelen 7 name jackett filetype 2 errors 4, no inode ref root 5 inode 4401772 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 114 namelen 9 name Downloads filetype 2 errors 4, no inode ref root 5 inode 5424640 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 39 namelen 36 name Community_Applications_VM_XML_Backup filetype 2 errors 4, no inode ref root 5 inode 5425855 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 40 namelen 3 name kde filetype 2 errors 4, no inode ref root 5 inode 5665745 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 43 namelen 11 name binhex-plex filetype 2 errors 4, no inode ref root 5 inode 5692894 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 44 namelen 11 name binhex-emby filetype 2 errors 4, no inode ref root 5 inode 5861581 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 148 namelen 21 name ESXi_Datastore_Lab_VM filetype 2 errors 4, no inode ref root 5 inode 6659751 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 50 namelen 9 name .htaccess filetype 1 errors 4, no inode ref root 5 inode 6659752 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 51 namelen 10 name index.html filetype 1 errors 4, no inode ref root 5 inode 6853111 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 163 namelen 6 name Backup filetype 2 errors 4, no inode ref root 5 inode 6873968 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 59 namelen 20 name appdata_ocevq7fsss5f filetype 2 errors 4, no inode ref root 5 inode 6874048 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 60 namelen 7 name .ocdata filetype 1 errors 4, no inode ref root 5 inode 6874049 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 61 namelen 5 name karlo filetype 2 errors 4, no inode ref root 5 inode 6874148 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 62 namelen 14 name files_external filetype 2 errors 4, no inode ref root 5 inode 6874252 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 63 namelen 3 name kna filetype 2 errors 4, no inode ref root 5 inode 6874344 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 64 namelen 3 name aig filetype 2 errors 4, no inode ref root 5 inode 6896982 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 66 namelen 17 name mariadb-nextcloud filetype 2 errors 4, no inode ref root 5 inode 7002239 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 68 namelen 14 name home-assistant filetype 2 errors 4, no inode ref root 5 inode 7063916 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 69 namelen 33 name Community_Applications_USB_Backup filetype 2 errors 4, no inode ref root 5 inode 7115094 errors 2001, no inode item, link count wrong unresolved ref dir 260 index 70 namelen 9 name nextcloud filetype 2 errors 4, no inode ref root 5 inode 7162647 errors 2001, no inode item, link count wrong unresolved ref dir 256 index 188 namelen 17 name TimeMachine_Karlo filetype 2 errors 4, no inode ref found 182835875840 bytes used err is 1 total csum bytes: 0 total tree bytes: 98664448 total fs tree bytes: 41385984 total extent tree bytes: 56918016 btree space waste bytes: 31632797 file data blocks allocated: 82555465728 referenced 13913096192 root@BGDunRAID:~#
October 3, 20178 yr Author 2 hours ago, johnnie.black said: btrfsfsck should only be run as a last resort, see here: 1 I am not a linux guy. I just want to make sure that I am doing this correctly. Right now, I have the SSD plugged in to my elementary OS laptop via USB, and it is still unmountable. It shows up in the GUI, but when I right click and select "mount", it will bark at me saying that it is unmountable. Is the command is "btrfs fsck" or "btrfs check --repair"? Can I do this on my elementary OS laptop or on my unRaid server? Do I need to go to maintenance mode or just boot up normally? Edited October 3, 20178 yr by pingmanping edit message
October 3, 20178 yr Community Expert All the instructions are on the link I posted, the SSD should be connect on the unRAID server.
October 3, 20178 yr Author 35 minutes ago, johnnie.black said: All the instructions are on the link I posted, the SSD should be connect on the unRAID server. Step 1 seems to be working. I am using Midnight Command to copy /x to my array. Once everything has been copied, what do I need to do with the cache disk? Do I need to format it as it shown in the Array Operation under the Main tab or do the "btrfs check --repair"? Is my SSD going bad? Edited October 3, 20178 yr by pingmanping
October 3, 20178 yr Community Expert Once all data is copied and checked you can format the SSD and restore the data. SMART looks fine, and most of these issues on SSDs are caused by a bad/intermittent connection, usually SATA cable, after as is well let the server work normally for a few days or so and check the log for any ATA errors, you can also post here new diagnostics if you need help analyzing them.
October 3, 20178 yr Author 11 minutes ago, johnnie.black said: Once all data is copied and checked you can format the SSD and restore the data. SMART looks fine, and most of these issues on SSDs are caused by a bad/intermittent connection, usually SATA cable, after as is well let the server work normally for a few days or so and check the log for any ATA errors, you can also post here new diagnostics if you need help analyzing them. Okay, so I don't need to do the "btrfs check --repair". I just need to format just like starting from scratch. Let me see if I understand you correctly. I can just move the copied files from the array to the newly formatted cache disk, correct?
October 3, 20178 yr Community Expert Just now, pingmanping said: Okay, so I don't need to do the "btrfs check --repair". I just need to format just like starting from scratch. Yes, when there are serious filesystem issues with btrfs it's better to format and start over. Just now, pingmanping said: I can just move the copied files from the array to the newly formatted cache disk, correct? Correct.
October 3, 20178 yr Author 34 minutes ago, johnnie.black said: Yes, when there are serious filesystem issues with btrfs it's better to format and start over. Correct. One more thing, I ran the Midnight Command via SSH, would this be a problem? I don't know if ssh will eventually timeout while I'm at work then messed up my file transfer. This one worries me more.
October 3, 20178 yr Community Expert It shouldn't timeout unless there's network or other error, but you can use screen to avoid those issues.
October 3, 20178 yr 6 minutes ago, pingmanping said: I don't know if ssh will eventually timeout while I'm at work then messed up my file transfer With MC you can start a copy operation in the background then it doesn't depend on SSH to stay connected.
October 3, 20178 yr Author 4 minutes ago, bonienl said: With MC you can start a copy operation in the background then it doesn't depend on SSH to stay connected. Okay. I just want to make sure. Does MC copy operation happen in the background by default? I really want to make sure because The reason I was asking was when I was doing rsync then my ssh somehow got disconnected, I had to do everything again. 10 minutes ago, johnnie.black said: It shouldn't timeout unless there's network or other error, but you can use screen to avoid those issues. Does unraid have the ssh timeout setup. I know this is different but whenever I login to my firewalls at work the firewalls had a default of 15 minutes of idle then after this it will auto-disconnect the ssh.
October 3, 20178 yr Community Expert Just now, pingmanping said: Does unraid have the ssh timeout setup By default it's always connected, I've done rsync transfers that lasted about a week.
October 3, 20178 yr 13 minutes ago, pingmanping said: Does MC copy operation happen in the background by default? When the copy window appears to choose source and destination, you should choose "background" instead of "Ok" to start the operation.
October 3, 20178 yr Author 33 minutes ago, bonienl said: When the copy window appears to choose source and destination, you should choose "background" instead of "Ok" to start the operation. I don't remember of doing that. I hope that my ssh won't get disconnected
October 3, 20178 yr Author 9 hours ago, johnnie.black said: By default it's always connected, I've done rsync transfers that lasted about a week. It seems like MC has finished but not sure if the copy is 100% completed. I got home and the ssh was disconnected. I formatted the cache disk via putting a check mark on the format under the Main tab. I don't know if this is an issue or what, but the data that I have on the cache disk didn't get erased after running the format. Then after the format, the cache disk got mounted. All my docker images, and some files are now back, I think.
Archived
This topic is now archived and is closed to further replies.