smeeee.again Posted March 27, 2019 Share Posted March 27, 2019 Hey. i have a problem with Unraid 6.6.7 (rolled back from 6.7.0-rc4) Whenever the SWystem performes a parity check at 60-80% the array chrashes. WebUI is available but faulty and the shares are shown via smb but can't be accesed. After reboot everything is fine. the problembs began after changing partiy disk to brand new 8TB device. the parity was build withoute error and the old drive became a datadisk within the array. The Diagnostics.zip after crash is nearly empty :-/ Hardware: CPU: Intel(R) Core(TM) i5-7600 MB: B250M-D2V Gigabyte RAM: Ballistix Sport LT 12GB Syba SI-PEX40064 4 Port SATA III PCI-EXPRESS 2.0 x 1 Controller 1x parity, 1 chache ssd, 7 Drives kartoffelhq-diagnostics-after crash.zip kartoffelhq-diagnostics-running normal.zip Quote Link to comment
trurl Posted March 27, 2019 Share Posted March 27, 2019 23 minutes ago, smeeee.again said: ... After reboot everything is fine. the problembs began after changing partiy disk to brand new 8TB device. the parity was build withoute error and the old drive became a datadisk within the array. ... Just to be safe, can you confirm that you rebooted after the new parity build and everything was fine? The reason I ask is because we need to be sure your new disk assignments were saved so it won't think your old parity (now data) disk is still parity. Can't tell much from those diagnostics, but the symptoms, including the empty diagnostics, suggest flash drive has been dropped. Put flash in your PC and let it checkdisk. While there make a backup. Make sure you use a USB2 port on your server for the flash. For future reference, you can always download a zipped backup of flash at Main - Boot Device - Flash - Flash Backup. 1 Quote Link to comment
smeeee.again Posted March 28, 2019 Author Share Posted March 28, 2019 Thanks for your help! 19 hours ago, trurl said: Just to be safe, can you confirm that you rebooted after the new parity build and everything was fine? The reason I ask is because we need to be sure your new disk assignments were saved so it won't think your old parity (now data) disk is still parity. I'm not shure, can i check this at some point? Workflow was: Addet new drive as second parity Drive -> let it run throug -> Parity check (everything working, 0 errors) -> made the new Drive the only parity drive (at this point i dont remember if i made a parity check) Or is it poissible to "delete" the parity Drive to build it new? 19 hours ago, trurl said: Put flash in your PC and let it checkdisk. While there make a backup. Make sure you use a USB2 port on your server for the flash. I checked the flashdrive with checkdisk without errors, the flashdrive is connected on the intern USB port via Adapter (https://smile.amazon.de/gp/product/B00DKWY0S8/ref=oh_aui_search_asin_title?ie=UTF8&psc=1) and works since i build the computer Quote Link to comment
trurl Posted March 28, 2019 Share Posted March 28, 2019 18 minutes ago, smeeee.again said: Workflow was: Addet new drive as second parity Drive -> let it run throug -> Parity check (everything working, 0 errors) -> made the new Drive the only parity drive (at this point i dont remember if i made a parity check) So are you saying you now have a disk assigned to the parity2 slot but nothing assigned to the parity slot? 1 Quote Link to comment
smeeee.again Posted March 28, 2019 Author Share Posted March 28, 2019 6 minutes ago, trurl said: So are you saying you now have a disk assigned to the parity2 slot but nothing assigned to the parity slot? No, the drive was assigned to the first slot, sry for not mention this. Quote Link to comment
trurl Posted March 28, 2019 Share Posted March 28, 2019 Still unclear. You said you added new drive as second parity drive. Did you not mean you put that new drive in the parity2 slot? 1 Quote Link to comment
smeeee.again Posted March 28, 2019 Author Share Posted March 28, 2019 1 minute ago, trurl said: Still unclear. You said you added new drive as second parity drive. Did you not mean you put that new drive in the parity2 slot? Yes, with old parity drive still in the first slot. Quote Link to comment
trurl Posted March 28, 2019 Share Posted March 28, 2019 You seem to have left out a lot of important details and it makes me think you have done all of this wrong. You had a disk in the parity slot. You put a new disk in the parity2 slot and built parity2. ..... Somehow you got from there to having the new parity2 disk in the parity slot and the old parity disk as a new data disk. Is that correct? Parity and parity2 are not the same and can't be exchanged. 1 Quote Link to comment
smeeee.again Posted March 28, 2019 Author Share Posted March 28, 2019 1 minute ago, trurl said: Is that correct? Yeah, this is correct... seems so I fucked this up pretty hard. is it possible to build the parity drive from scratch without data loss? Quote Link to comment
trurl Posted March 28, 2019 Share Posted March 28, 2019 You shouldn't lose any data since parity doesn't have any data and all of the data disks are independent. I guess it really doesn't matter at this point exactly how you got to where you are. Rebuilding parity is what you need to do and maybe even that is what it was doing when you said it was doing a parity check since if you put a new disk in the parity slot it is going to want to rebuild parity. Did you have any data on that new data (old parity) disk yet? 1 Quote Link to comment
smeeee.again Posted March 28, 2019 Author Share Posted March 28, 2019 5 minutes ago, trurl said: Did you have any data on that new data (old parity) disk yet? yes, it is unfortunately already in use. Quote Link to comment
trurl Posted March 28, 2019 Share Posted March 28, 2019 That's OK. The reason I was concerned about whether your new disk assignments had been saved successfully was because if it thought that disk was still parity it would overwrite its data with parity. If you still have that flash drive out, put it in your PC and edit config/disk.cfg to set autostart to no so you can boot up and check your disk assignments. 1 Quote Link to comment
smeeee.again Posted March 28, 2019 Author Share Posted March 28, 2019 5 minutes ago, trurl said: If you still have that flash drive out, put it in your PC and edit config/disk.cfg to set autostart to no so you can boot up and check your disk assignments. I can't find the point autostart in the file. Do you mean startArray? I changed it from "yes" to "no" and started the server. The hard disks are all where they belong. Is a rebuild still triggered that way? Quote Link to comment
smeeee.again Posted March 28, 2019 Author Share Posted March 28, 2019 8 minutes ago, trurl said: Post new diagnostics new diagnostics.zip Quote Link to comment
trurl Posted March 28, 2019 Share Posted March 28, 2019 Looks OK. Since you haven't started the array yet, you can just skip the first step Stop array Unassign disk Start array with disk unassigned Stop array Reassign disk Start array to begin rebuild (In this case, "disk" refers to your parity disk, but those instructions are the same for rebuilding a data disk to the same disk since it is all the same parity calculation.) Quote Link to comment
smeeee.again Posted March 28, 2019 Author Share Posted March 28, 2019 Thanks for the help! I do a parity check after the rebuild and then report again. Quote Link to comment
trurl Posted March 28, 2019 Share Posted March 28, 2019 If it seems to be going wrong try to get another diagnostic. If you look at Main - Array Devices, it should have a lot of writes to parity, a lot of reads of all other disks, and zero errors on all disks. Quote Link to comment
smeeee.again Posted March 29, 2019 Author Share Posted March 29, 2019 The rebuild just finished without errors. Now it looks like this in webui. Docker Services and the VM are running. Quote Link to comment
trurl Posted March 29, 2019 Share Posted March 29, 2019 Post diagnostics. If the webUI won't let you get them you can try to get the from the command line as explained here: https://forums.unraid.net/topic/37579-need-help-read-me-first/ Quote Link to comment
smeeee.again Posted March 29, 2019 Author Share Posted March 29, 2019 I can't get a diagnostics file via the webUI. Via the console I get the error message from the screenshot. Now I tried to capture the syslog, but to get it I have to shut down the server and read the USB stick on my computer. At the bottom you can see how the messages "Array undefined" and "Array Started" alternate in different intervals. Quote Link to comment
Squid Posted March 29, 2019 Share Posted March 29, 2019 What is the output of df -h Quote Link to comment
smeeee.again Posted March 29, 2019 Author Share Posted March 29, 2019 3 minutes ago, Squid said: What is the output of df -h root@KartoffelHQ:~# df -h Filesystem Size Used Avail Use% Mounted on rootfs 5.8G 5.8G 0 100% / tmpfs 32M 236K 32M 1% /run devtmpfs 5.8G 0 5.8G 0% /dev tmpfs 5.9G 0 5.9G 0% /dev/shm cgroup_root 8.0M 0 8.0M 0% /sys/fs/cgroup tmpfs 128M 884K 128M 1% /var/log /dev/sda1 30G 402M 29G 2% /boot /dev/loop0 8.2M 8.2M 0 100% /lib/modules /dev/loop1 4.9M 4.9M 0 100% /lib/firmware /dev/md1 3.7T 3.1T 563G 85% /mnt/disk1 /dev/md2 3.7T 3.1T 604G 84% /mnt/disk2 /dev/md3 4.6T 4.0T 620G 87% /mnt/disk3 /dev/md4 4.6T 4.1T 552G 89% /mnt/disk4 /dev/md5 4.6T 2.3T 2.3T 50% /mnt/disk5 /dev/md6 3.7T 711G 3.0T 20% /mnt/disk6 /dev/md7 5.5T 11G 5.5T 1% /mnt/disk7 /dev/sdf1 239G 187G 52G 79% /mnt/cache shfs 31T 18T 13T 57% /mnt/user0 shfs 31T 18T 14T 57% /mnt/user /dev/loop2 30G 19G 11G 64% /var/lib/docker /dev/loop3 1.0G 17M 905M 2% /etc/libvirt shm 64M 0 64M 0% /var/lib/docker/containers/89466d7ab026469d123e4ba4a13c0ee3163565b62ee4192cc0a300e297511fd2/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/9e9c6a6a0422683abc65ab76b338e2bf5cb40e157081cdfd717b6c5d7a8cc62c/mounts/shm shm 64M 8.0K 64M 1% /var/lib/docker/containers/3c081b4b83ec61bb8d5bb09b6f841c7148f633c8c4e2cea0011ed04424dffae7/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/05f143591df703c25bbfb1cc4161157e32415ee6e8e658b8b17000613c308ba6/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/aa2c6ac117296e52339feed1afaf51b01598c216489bfb13e7d452432c69cb32/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/7f409b380de76f8a41ca237cc8d3ebd13df2955cfd17b341485a437dd6ec5aa5/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/e83688f60259f94fb25cd06a4f97841f71fcb6f0d81bce36ccc7f94bce107969/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/01b2dd903585a67a0e33ca24857538b3651e107fde03e6727961c13e9760e7b4/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/267de35ba0fc79b147a96475186cd10e3136036fb16b991cb4253988e549a723/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/e6ad761dbd49b663c38a3c1e822e8da16f16e399c9c133375029bbb65e671844/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/8bbf7613bd879d9c29b0d53f2245e29e2ed858aaa9c72d993f3286b85c177aae/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/b74a0e0c2b90fef37bf683085cb1e117c8173a3daf49db809e99a8572673fab9/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/03ed9eea62d0dc11dc1a74ffc10ecfab7ae285b8d14cec8000e49d664797ffe4/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/e67eab68be52d12d55f9ba2c95483589a90e7c447f4736ec0c077ac726e8533c/mounts/shm shm 64M 0 64M 0% /var/lib/docker/containers/3413a746470808c88aa678f661bec03c67d5bef4018dd476beb3030cd5b32994/mounts/shm Quote Link to comment
Squid Posted March 29, 2019 Share Posted March 29, 2019 1 minute ago, smeeee.again said: rootfs 5.8G 5.8G 0 100% / This is going to play havoc with your system. Most likely a misconfigured docker container is writing to RAM instead of a share (ie: writing to /mnt/appdata instead of /mnt/user/appdata) Your only recourse though right now is to reboot powerdown -r and then investigate Quote Link to comment
smeeee.again Posted March 29, 2019 Author Share Posted March 29, 2019 The Docker Containers are configured correctly. Most of them have been running without errors for some time and the newest ones are deactivated to exclude them as a source of errors. The problem only seems to occur during a parity check or, as it turns out, during the rebuild. To see if this is really the case, I will run the server for a few days without a parity check, unless there are other ideas. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.