salvdordalisdad Posted May 19, 2022 Share Posted May 19, 2022 (edited) Hi Guys, I just recovered my 3rd unraid server ("Plus" license) & replaced a couple of disks intending it to become a backup target. So I zeroed all the disks & created a new array of 7 x 4Tbyte disks, with one of them as parity. All the disks are completely blank, freshly formatted in xfs. These were happy in the old servers, so I assumed they'd be OK for this job. However, it seems not. The parity build (1st time as it's a new array) says it will take 25 days to complete. When it first started, it said 10 hours, and has crept up since then. So clearly something's wrong, probably a dying disk, but I don't know how to diagnose it, so please can someone read the diags & point me in the direction of the faulty disk?n3-diagnostics-20220519-1514.zip Thanks in advance. Much appreciated. Lee Edited May 19, 2022 by salvdordalisdad Quote Link to comment
JorgeB Posted May 19, 2022 Share Posted May 19, 2022 There's nothing logged, I would suggest canceling the sync and running a test with the diskspeed docker to confirm all disks are performing normally, also make sure this high CPU utilization is expected: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 27587 root 20 0 16452 15688 2888 R 96.2 0.1 240:12.17 port_ping+ 30062 root 20 0 6004 4060 2712 R 92.3 0.0 0:01.02 lsof Quote Link to comment
salvdordalisdad Posted May 19, 2022 Author Share Posted May 19, 2022 (edited) So I did some digging into those diags & saw a LOT of dead tcp connections, and spotted that port_ping+ I cancelled all of the (legacy from previous config) user scripts & rebooted. Now those connections are gone, which is as expected. The parity build time has gone down to 9 days - which is an improvement - woohoo! I think I've spotted another slight issue... Can't run docker diskspeed Docker ... "path does not exist" There was no /mnt/system/docker folder, so I created one - now it doesn't complain about that anymore. There's no /mnt/user/appdata folder - but I can't create one of those... "no medium found" Contents of /mnt/user: root@n3:/mnt# cd /mnt/user root@n3:/mnt/user# ls -l total 0 drwxrwxrwx 1 nobody users 6 May 19 11:07 filestore/ I'd expected a bunch of folders, system, appdata, domains... Do I just create those, or is something lower level going to be required to fix that? - open to suggestions. Tried creating "system" as a share - didn't do anything, no folder in /mnt/user at all. So I created the folders /mnt/system/docker and /mnt/user/filestore/appdata But even with those fudged the docker refused to start. Definitely something lower level needed. I will tiidy up that mess I just made...and await the right instructions. (Even if those instructions involve starting from scratch)_. Thanks in advance Lee Edited May 19, 2022 by salvdordalisdad Quote Link to comment
JorgeB Posted May 19, 2022 Share Posted May 19, 2022 23 minutes ago, salvdordalisdad said: There's no /mnt/user/appdata folder - but I can't create one of those... "no medium found" Doe it work if you type for example mkdir /mnt/disk1/appdata Quote Link to comment
salvdordalisdad Posted May 19, 2022 Author Share Posted May 19, 2022 Hmm, yes it dort of works, but it still says "path does not exist". I think I'm going to scratch this server & start again. Now I need to find out how to do that on the existing USB drive... some searching to do I think. But not right now, out of time, have to be after the weekend now. ;-/ Thanks for the nudges, much appreciated... Quote Link to comment
trurl Posted May 19, 2022 Share Posted May 19, 2022 45 minutes ago, salvdordalisdad said: /mnt/system/docker This folder would be in RAM since it is not a disk path or a user share path. docker.cfg in those earlier diagnostics has a path in /mnt/cache for your docker.img, since you don't have cache that path is also in RAM. Quote Link to comment
trurl Posted May 19, 2022 Share Posted May 19, 2022 And appdata, domains, system shares are configured cache-only 1 Quote Link to comment
JorgeB Posted May 19, 2022 Share Posted May 19, 2022 8 minutes ago, trurl said: And appdata, domains, system shares are configured cache-only Yeah, that makes sense, since there's no cache it generates an error. Quote Link to comment
salvdordalisdad Posted May 20, 2022 Author Share Posted May 20, 2022 That's really interesting, but if I have no cache? Anyway, I started with a fresh 6.10 trial on a new USB, as I had clearly got something wrong & I am comfortable with starting from scratch. So I ran the diskspeed benchmarks & was a little surprised at the max & min values. disk MB/sec MB/sec sdf 200 97 sdg 157 70 sdh 200 87 sdb 150 73 sdc 200 94 sdd 183 84 sde 142 67 So I chose the fastest (sdc) as the parity. Are these figures good enough? Do they indicate any underlying issues? Anyway, just rebuilt the array again & started it & it's now saying 16 days . The CPU is trundling along at 3% util, so it's not that. All the SATA ports are 6Gbps, all the disks are 6Gbps. Any suggestions, or just "yeah it might do that" Thanks in advance for any pointers. Also - when I want to move back to the old USB, what do I copy from this USB drive to that one & what do I delete on that one? (I know that the license in in the plus.key file, but virtually naff all else of what's on there). Quote Link to comment
trurl Posted May 20, 2022 Share Posted May 20, 2022 1 hour ago, salvdordalisdad said: That's really interesting, but if I have no cache? That was entirely our point. You have no cache, but many things were configured to write to that cache you don't have. 1 hour ago, salvdordalisdad said: now saying 16 days Attach new diagnostics to your NEXT post in this thread. Quote Link to comment
JorgeB Posted May 20, 2022 Share Posted May 20, 2022 16 hours ago, JorgeB said: and running a test with the diskspeed docker to confirm all disks are performing normally Besides new diags see if you can run this now. Quote Link to comment
salvdordalisdad Posted May 20, 2022 Author Share Posted May 20, 2022 Hi JorgeB Thanks, I did a quick & dirty single disk array & got docker up & running on fresh demo license & new USB stick (seemed quickest!) These are the stats from the disks: disk MB/sec MB/sec sdf 200 97 sdg 157 70 sdh 200 87 sdb 150 73 sdc 200 94 sdd 183 84 sde 142 67 They all seem pretty much within a similar range, no single one stands out as clearly having difficulties. Any thoughts? Thanks in advance. (Nice picture - Slava Ukraine! ) Quote Link to comment
JorgeB Posted May 21, 2022 Share Posted May 21, 2022 Those numbers look fine, please post new diags during parity sync. Quote Link to comment
Solution salvdordalisdad Posted May 26, 2022 Author Solution Share Posted May 26, 2022 Hi All, Rather than fret about it, I figured I'd just ask it every few days & see. I asked it today & found this message: Last check completed on Tue 24 May 2022 08:04:42 AM BST (two days ago) Duration: 4 days, 7 hours, 14 minutes, 8 seconds. Average speed: 10.8 MB/s Finding 0 errors And it's all running fine, no errors. Pesky calculator got it massively wrong! Oh well, it was a very ueful learning experience - stop fretting & let it do its thing! Thanks to all who chipped in. Quote Link to comment
salvdordalisdad Posted May 28, 2022 Author Share Posted May 28, 2022 ...and a post scriptum for the record... After the parity build had completed, the system was left idle for a few more days, adjusting this & adding that etc. The parity drive started giving READ errors, only a few hundred, but even a handful is too many for a fresh build. So I swapped it out - reluctantly expecting another 5 - 25 day parity build - and was shocked to see the estimate for the new parity build should take around 8 hours... It actually took 10 hours, but heavens to betsy what a difference! Just a shame that the original cause (the Toshiba 7200 rpm) parity drive problem could not be spotted earlier...I had my suspicions but no evidence. So what have we learned here today? If the parity build says it's going to take 25 days - one of the disks is probably faulty. Now go & figure out which one! Quote Link to comment
trurl Posted May 28, 2022 Share Posted May 28, 2022 1 hour ago, salvdordalisdad said: So what have we learned here today? If the parity build says it's going to take 25 days - one of the disks is probably faulty. Now go & figure out which one! After you have already ruled out other problems. Bad connections are much more common than bad disks. Diagnostics can often tell what the problem is, but don't just assume that slow parity checks are bad disks. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.