Performance problem

[thany]

I started copying stuff to my array, via the network. The network is fine, it's a 10Gb connection, and I was able to write at about 60MB/s. Not incredible for 10Gb, but at least it was going steadily. The share being copied to is not backed by any cache pool, and is set to "high water" mode. The source being copied from, can supply a steady 100MB/s.

 

So I wonder:

 

• why is it seldom getting above 1 whole MB/s, but only after having copied a few TB's?
• why does this need 100% CPU on a single core (sometimes multiple, for fun I guess) on the unRAID server? What is it *doing*??
• why is it not utilising my harddisks basically at all? (they're effectively idle according to the dashboard)
• why can it go quickly at the start? And then gradually degrade to such an extend? This makes no sense to me.

 

Mostly I wonder, what the hecko do I do to make it go as fast as it should be able to go, please.

[thany]

Well, the mover is running.

 

Okay.

 

But why does that get *so* much I/O priority? And can't it use one of the other disks?

Mover should probably behave so that it always "gets out of the way". It's a background operation, and actually using the array for it's apparant intended use (being a NAS) is a foreground operation which should (hopefully obviously) take precedence, not yield, to background processes.

 

But is it the mover? Don't know. It's hard to know for sure what is doing what.

[thany]

I've set the sahre to "most-free" so that the mover is moving to the cached share that is still on "high water" mode.

Now at least I can sort of resume copying to the share in question. Other disks have spun up, as they should. They're there to help! Use them!

 

Not that it's going well now, mind you. It's still circling around 20MB/s 5MB/s 2MB/s (yeah it's degrading again), and stopping for many seconds on end intermittently.

 

How can I make to GO? I've never had this type of problem with previous NAS systems. They just just worked theur arses off to their absolute maximum potential. They wouldn't just wait around, or utilise one drive to be bombarded with absolutely bloody everything while the others are resting on the bahamas.

 

Please, mister unraid, fix this. unRAID is basically unusable in this state.

Edited November 20, 2022 by thany

[thany]

One more update then.

 

Mover is done. CPU utilisation is stil abnormally high, but I can't figure out WHAT is doing that. Either way, now that the mover is done, and all harddisks have eventually spun up, you'd expect things to go quicker, wouldn't you?

 

Well no. Even readin from the array gets me an excruciating 200KB/s if I'm "lucky". Writing to the array... Still the same problem.

 

What is happening?! Why is everything so slooooooooow? What is unRAID *doing*?

[thany]

Last update for the day.

 

Gave it a reboot. Performance is back to acceptable levels.

 

But why? Why why why? Why was it so incredibly difficult to get a harddisk, that can deliver 180MB/s do pull more than 200KB/s from it?

I'm getting mentally ready to be profesionally done with unRAID. Al these absolutely monstrous weirdnesses shouldn't exist in a commercial product, especially based on a good foundation.

 

Seriously, I NEED to know what's going on in order to evaulate if unRAID was a bad choice, or if it can be fixed so it won't happen again.

[thany]

Next day. Performance is degrading again. It's not doing anything in the background. Please provide a solution.

[Vr2Io]

Any SMR disks in array ?

[JorgeB]

Please post the diagnostics.

[thany]

4 hours ago, JorgeB said:

Please post the diagnostics.

Totally forgot. Here you go. These are not gathered while the problem is ocurring though. Just while the system is more-or-less idling I/O-wise. Let me know if you need more.

 

8 hours ago, Vr2Io said:

Any SMR disks in array ?

I don't actually know. I was hoping 8TB are not produced as SMR, but only the bigger varieties like 16TB and up.

 

EDIT: ahh crap, they might all be SMR. That's the "bad" kind of technology, right? Those fuckers! It's not neccesary for 8TB drives.

Sorry to make this topic drag on. Sorry for trying to blame unRAID. Sorry for my tone. It might be my own fault, kind of. Well ,Seagate's fault afaic

 

unraid-diagnostics-20221121-1621.zip

Edited November 21, 2022 by thany
SMR bollocks :(

[JorgeB]

Yes, those disks are SMR and known to in some cases slow down by a lot during writes.

[Vr2Io]

21 minutes ago, thany said:

That's the "bad" kind of technology, right?

Sometimes it can very bad, I will avoid any SMR.

 

Unraid array performance won't that bad level in general.

Edited November 21, 2022 by Vr2Io

[thany]

I've just ordered new ones. Toshiba MG08 512e, I hope these are good. I filtered out SMR disks while looking for them and my supplier explicitly says they're meant for NAS. Fingers crossed, as we live in strange times.

 

Again sorry for this debacle. Perhaps a suggestion if I may: build in some kind of check for SMR, and display a (dismissable) warning when SMR is detected. Not sure if it can be detected though.

[JorgeB]

Some SMR disks work fine if it's mostly large file transfers, some models are always much slower, in my experience most Toshiba SMR models, Seagate are usually fine, again for large files, but that specific model seems to be hit and miss.