Hey all! I recently upgraded my Unraid NAS to an SSD cache finally, but my speeds are still at around 80MBps write. My read speed on the other hand is still maxing out my 1Gpbs connection at 112MBps like you would expect. Do any of you know why my 500MBps SSD cache write speed isn't completely saturating my 1Gbps connection? I feel like there is something obvious that I am missing here. Thanks!

Are you sure you are writing to cache? Default settings for user shares are to not cache. 

4 minutes ago, trurl said:

Are you sure you are writing to cache? Default settings for user shares are to not cache. 

Yes I checked that, and all my shares have the cache pool enabled

What are you writing to the cache?  Thousands of small files or a few massive BluRay size iso's.  Remember that there is file overhead on both ends of a file transfer. 

 

I do seem to recall that the file overhead on Unraid is substantial when writing to a Share.  I do notice this when I do my monthly Data backups of my Windows Documents folder as compared to the transfer of DVD and BluRay ISO's.  (The BluRay speeds are even slightly faster than the DVD ISO's!)  In fairness, I don't have the fastest SSD's for my cache drives. 

Edited May 27, 20215 yr by Frank1940

2 hours ago, Frank1940 said:

What are you writing to the cache?  Thousands of small files or a few massive BluRay size iso's.  Remember that there is file overhead on both ends of a file transfer. 

 

I do seem to recall that the file overhead on Unraid is substantial when writing to a Share.  I do notice this when I do my monthly Data backups of my Windows Documents folder as compared to the transfer of DVD and BluRay ISO's.  (The BluRay speeds are even slightly faster than the DVD ISO's!)  In fairness, I don't have the fastest SSD's for my cache drives. 

I've tried a 4GB MKV, a folder with a bunch of little audio files, and a folder with a bunch of random files in it so I think I've eliminated the possibility of that. This might be a big newb question, but what do you mean by file overhead? lol. Do you mean there is a process that is happening in Unraid behind the scenes when a file is written? 

Absolutely!  On the receiving end (Unraid, in this case), the file system must create an entry in its file allocation table for the file name.  It must decide where to begin writing the file on the physical disk.  It has to keep track of all the sectors where it writes the actual file and update the file's allocation table to reflect this information.   (It is my understanding that Unraid has a proprietary file writing software for shares because it must also determine which disk to use to write the file to as well as where to write the information on that chosen disk.)

 

On the sending end, the copy program has to go to the File allocation table, to find out where the file is stored.  Once it has this information, (assuming that the file is on a HD), the heads have to relocate to those tracks on the physical disk to read the information. 

 

There is constant handshaking going on between both systems to assure that the transfer handled in an orderly fashion.  (Unraid does use RAM as a buffer to store incoming data but large transfer will usually over run this RAM buffer in a very short period of time.  You will often see this as a burst of +100MB/s transfer at the beginning of the copy operation.  Once this buffer is full, the transfer will slow down to what the disk can handle. 

 

One more thing.  Many SSD's can read faster than they can write.  (Be careful of 'specmanship' on manufacturer's claims of drive performance.  It will always be the best case performance, not the worst case...)

6 hours ago, Frank1940 said:

Absolutely!  On the receiving end (Unraid, in this case), the file system must create an entry in its file allocation table for the file name.  It must decide where to begin writing the file on the physical disk.  It has to keep track of all the sectors where it writes the actual file and update the file's allocation table to reflect this information.   (It is my understanding that Unraid has a proprietary file writing software for shares because it must also determine which disk to use to write the file to as well as where to write the information on that chosen disk.)

 

On the sending end, the copy program has to go to the File allocation table, to find out where the file is stored.  Once it has this information, (assuming that the file is on a HD), the heads have to relocate to those tracks on the physical disk to read the information. 

 

There is constant handshaking going on between both systems to assure that the transfer handled in an orderly fashion.  (Unraid does use RAM as a buffer to store incoming data but large transfer will usually over run this RAM buffer in a very short period of time.  You will often see this as a burst of +100MB/s transfer at the beginning of the copy operation.  Once this buffer is full, the transfer will slow down to what the disk can handle. 

 

One more thing.  Many SSD's can read faster than they can write.  (Be careful of 'specmanship' on manufacturer's claims of drive performance.  It will always be the best case performance, not the worst case...)

Thank you! That makes sense. So does that mean that I might not ever get anything higher than 80MBps write, but my read speed could continue to increase if I had a 10Gbps network? I want to upgrade to 10Gbps network in our new house we are moving to, but I just want to make sure that I'll be able to take advantage of it since it is expensive. Also I've tested my SSD drive I use for my cache, and it is definitely capable at writing faster than 80MBps, but I'm figuring that it might be the overhead you mentioned coming from Unraid that is the bottleneck here. Write speeds aren't a super big deal for me as I'm usually reading from the NAS 90% of the time anyway, but I just want to make sure that I'm not losing out on performance due to my lack of knowledge. 

With 10Gbs, transfer speed will the be dependent of the drive(s) involved in the read(/write) operation.  Remember with Unraid, the entire file is stored on one drive.  So the transfer speed of any transfer is limited by the read speed of that drive.  Of course, if one were doing multiple file transfers and those files were on different physical drives then the 10Gbs would be advantage.  (Provided the receiving client(s) could handle the data transfer at that rate on the receiving end.) 

 

If you have a situation with multiple clients, you might want to consider a 10Gbs network but be sure that look at the cost of upgrading the entire network to 10Gbs.  The last time I looked (granted some time ago), 10Gbs switches were not cheap!

 

There is a 2.5Gbs Ethernet networking standard out now that is probably more affordable.  (I knows that there are several MB's that have 2.5Gbs NIC as the standard NIC.  Linux has the driver for those NIC's included in the standard build.   You might want to look at that first...

Edited May 27, 20215 yr by Frank1940

3 hours ago, Frank1940 said:

With 10Gbs, transfer speed will the be dependent of the drive(s) involved in the read(/write) operation.  Remember with Unraid, the entire file is stored on one drive.  So the transfer speed of any transfer is limited by the read speed of that drive.  Of course, if one were doing multiple file transfers and those files were on different physical drives then the 10Gbs would be advantage.  (Provided the receiving client(s) could handle the data transfer at that rate on the receiving end.) 

 

If you have a situation with multiple clients, you might want to consider a 10Gbs network but be sure that look at the cost of upgrading the entire network to 10Gbs.  The last time I looked (granted some time ago), 10Gbs switches were not cheap!

 

There is a 2.5Gbs Ethernet networking standard out now that is probably more affordable.  (I knows that there are several MB's that have 2.5Gbs NIC as the standard NIC.  Linux has the driver for those NIC's included in the standard build.   You might want to look at that first...

Yah I have looked at the 2.5Gbps standard, and I might end up going with that. It just depends on how much the wallet will allow. I should see an increase of read write speed though shouldn’t I if I have an Samsung Evo SSD cache drive, and I upgrade my network? The way that I understand it, this should be a yes, but my write speeds are capped at roughly 80MBps so I’m not thinking I’ll get much of an increase there.

One thing to think about is that writes tend to be a background operation in the normal day-to-day operation of the modern OS. MS introduced this 'feature' back in the 1990's and I don't think they were the first.  Basically, the writes were buffered to memory and actually performed when the OS had some idle CPU cycles.  (As I remember, computers and OS's of that era basically became unresponsive when disk IO was being performed and this was one way to address that problem.  Plus, the disk in question could be a floppy disk and a file write could take from many seconds to minutes!   I think the max delay time was many seconds but my memory may be playing tricks.  What I do remember is that many of us turned this feature off because that version of Windows so unstable, it could crash before the write finished.)  So slow writes can be an non problem unless, for some reason, you need to wait for them to finish. 

 

Plus, there are many ways that read responsiveness can be improved since it can be predicted what data might be requested next and disk drive could read that data into its onboard RAM while the drive read head is in position to read it before it is even requested!

 

There is one thing I have not mentioned.  The IO part of Unraid (as I understand) runs in a single core of the CPU.  Thus, the passmarks (a CPU benchmark) of the individual core can be bottleneck if that core becomes saturated.  Example, if a four core CPU and six core CPU have the same total CPU passmark benchmark, the four core CPU  could have superior IO performance.  I am reasonably certain that the file creation overhead is fairly CPU intensive one because the file systems used by Unraid do have considerable redundancy so that disk recovery operations can have a reasonable chance of success.

Ok thank you! I appreciate your help