CPU spikes/slow response after rebuild, using unassigned drives

[vonfrank]

Hi, 

 

I just started rebuilding my two unraid servers which have been running on different locations so far, but recently I have decided to "combine" the two servers. The two servers both consist of 8 threaded Xeon CPUs, with one on a ATX asRock rack motherboard, and the other on a MITX asRock rack motherboard, both with 32gb RAM.

 

I want to have the ATX/DDR3 server handling storage, and the MITX/DDR4 running as a node, consuming the data provided by the storage server.

 

The storage server is running a LSI sas card to provide sata ports for my 8x Sata HDDs, while I will put my 2x parity HDDs on the onboard raid controller, as well as the 4x 250gb samsung SSDs.

 

The node has 2x 500gb samsung SSDs as the only storage medium, as I expect the node to consume data from the storage server.

 

My question is, how do I maximise the link between the two servers? At the moment a 1gbE/112mb/s link will be sufficient, but I don't always see constant 112mb/s transfer speed between VMs on the servers.

 

I have also tried using unassigned drives, to have dockers consuming the data from the storage server. If the node server is transferring more than 10/15mb/s to the storage server, I see some very high CPU spikes, and the webUI/entire server becomes very unresponsive/slow for some time, like some kind of cache/buffer stacks up. I would suspect unassigned drives to use some kind of temporary storage space, while transferring the data from this folder over time to the network drive it becomes filled up and has to use all CPU power to empty this storage space, but my 2 SSDs are not near full (10% of 1tb and the moment).

 

What would be the ideal way to setup my systems in this setup? Would a 10gbe network card be the only way to improve this?

 

Another thing I would like to know, is what would be the optimal setup for my two samsung SSDs in my node server. I would like to have a "Raid 0" setup, with just 1tb storage from my 2x 500gb ssds, and the performance from 2 SSDs in raid 0, as they are not storing data, only hosting VMs and docker containers. I have them in a BTRFS array, as the main array, but I only see one of the SSDs filling up. Would it be better to put a "dummy" SSD/HDD in the array and the 2x SSDs in the cache, or simply put my SSDs in the array with one in parity (even though parity is not needed for this setup).

[dlandon]

UD mounts devices but has no control over how mounted devices are used.  That is up to Linux.  What you are probably seeing is the disk buffering.  Install this plugin:

and adjust the 'Disk Cache' values.  These are Linux values that control disk buffering.  Read the guide on suggested values.

 

If you have a large amount of ram in your server, these values default to a very large value because they are a percentage of all available ram.

[vonfrank]

20 hours ago, dlandon said:

UD mounts devices but has no control over how mounted devices are used.  That is up to Linux.  What you are probably seeing is the disk buffering.  Install this plugin:

and adjust the 'Disk Cache' values.  These are Linux values that control disk buffering.  Read the guide on suggested values.

 

If you have a large amount of ram in your server, these values default to a very large value because they are a percentage of all available ram.

Thanks a lot dlandon, I already have the plugin on both servers, I will do some reading on the disc buffering values. Do you have a suggestion with my SSD array? Is this a good way to go, or should I move the SSDs to the cache instead, or move one of the SSDs into parity?

[dlandon]

It is not recommended to put SSDs in the array.  Your SSD would be better suited as a cache drive.

[vonfrank]

4 hours ago, dlandon said:

It is not recommended to put SSDs in the array.  Your SSD would be better suited as a cache drive.

I know, but as far as i know, i am not able to start my server with only cache drives. It is required to put at least one drive in the array, and I would like to have a SSD-only server?

[JonathanM]

1 hour ago, vonfrank said:

I know, but as far as i know, i am not able to start my server with only cache drives. It is required to put at least one drive in the array, and I would like to have a SSD-only server?

You can toss in any old USB stick (not your unraid boot drive) as Disk1 and assign all your SSD's to the cache pool or UD.

[dlandon]

1 hour ago, vonfrank said:

I know, but as far as i know, i am not able to start my server with only cache drives.

Correct, you need at least one array disk.

 

1 hour ago, vonfrank said:

I would like to have a SSD-only server?

One last time, it is not advisable and you need to stop asking about it.  The answer will not change.