Luke Mazzotta Posted March 26, 2020 Share Posted March 26, 2020 I was wondering if there is anyway to use my RAM as a read and write cache in Unraid 6. Quote Link to comment
testdasi Posted March 26, 2020 Share Posted March 26, 2020 5 hours ago, Luke Mazzotta said: I was wondering if there is anyway to use my RAM as a read and write cache in Unraid 6. RAM is always used by Linux (including Unraid which based on Linux) to cache write. If the cached data is still in RAM then it is also used automatically as read cache. There is no other functionality to use RAM as other forms of read/write cache 1 Quote Link to comment
aerosmith9110 Posted May 18, 2020 Share Posted May 18, 2020 I have the same problem. I have 300+gb ram and with 10gbe network and I am getting only 100MBps + of transfer speed basically a little more than if I had 1gbps connection. checked the server and client side both getting 10gbps and same goes to the trunks. I made sure that I only have the 10gbe links active. Current Disk Cache Settings vm.dirty_background_ratio = 10 % vm.dirty_ratio = 80 % I don't see that the ram is being used using free -m I see cache did increase when I transferred but speed is still around 100MBps root@Jupiter:~# free -m total used free shared buff/cache available Mem: 387014 866 385251 796 895 383676 Swap: 0 0 0 root@Jupiter:~# free -m total used free shared buff/cache available Mem: 387014 853 379108 796 7052 383237 Swap: 0 0 0 I did do a lot of searches and thus tried the above settings. Maybe someone can help? Not a linux person. Quote Link to comment
JorgeB Posted May 18, 2020 Share Posted May 18, 2020 Run a single stream iperf test to check network bandwidth. Quote Link to comment
aerosmith9110 Posted May 18, 2020 Share Posted May 18, 2020 1 hour ago, johnnie.black said: Run a single stream iperf test to check network bandwidth. Thanks for the tip. I am checking mtu right now. and doing some adjustments is making it better, but I will need to figure out local vs internet traffic for the mtu. I will do my research as well. But I would not complain if you point me to the right direction. Quote Link to comment
aerosmith9110 Posted May 18, 2020 Share Posted May 18, 2020 I am now getting root@Jupiter:~# iperf3 -s ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 192.168.1.145, port 56563 [ 5] local 192.168.1.105 port 5201 connected to 192.168.1.145 port 56564 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 370 MBytes 3.10 Gbits/sec [ 5] 1.00-2.00 sec 379 MBytes 3.18 Gbits/sec [ 5] 2.00-3.00 sec 385 MBytes 3.23 Gbits/sec [ 5] 3.00-4.00 sec 378 MBytes 3.17 Gbits/sec [ 5] 4.00-5.00 sec 380 MBytes 3.19 Gbits/sec [ 5] 5.00-6.00 sec 380 MBytes 3.19 Gbits/sec [ 5] 6.00-7.00 sec 384 MBytes 3.22 Gbits/sec [ 5] 7.00-8.00 sec 382 MBytes 3.21 Gbits/sec [ 5] 8.00-9.00 sec 387 MBytes 3.24 Gbits/sec [ 5] 9.00-10.00 sec 382 MBytes 3.21 Gbits/sec [ 5] 10.00-10.04 sec 14.3 MBytes 3.23 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.04 sec 3.73 GBytes 3.19 Gbits/sec receiver still doing some tweaks. basicall mtu=9000 Quote Link to comment
JorgeB Posted May 18, 2020 Share Posted May 18, 2020 Better, but when all is working good would should get close to line speed with iperf. Quote Link to comment
aerosmith9110 Posted May 18, 2020 Share Posted May 18, 2020 33 minutes ago, johnnie.black said: Better, but when all is working good would should get close to line speed with iperf. Is this normal? sometimes it gives me consistent speed. The server isn't doing anything. Accepted connection from 192.168.1.145, port 51707 [ 5] local 192.168.1.105 port 5201 connected to 192.168.1.145 port 51708 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 365 MBytes 3.06 Gbits/sec [ 5] 1.00-2.00 sec 372 MBytes 3.12 Gbits/sec [ 5] 2.00-3.00 sec 1.59 MBytes 13.4 Mbits/sec [ 5] 3.00-4.00 sec 1.70 MBytes 14.3 Mbits/sec [ 5] 4.00-5.00 sec 1.94 MBytes 16.3 Mbits/sec [ 5] 5.00-6.00 sec 2.41 MBytes 20.2 Mbits/sec [ 5] 6.00-7.00 sec 2.09 MBytes 17.5 Mbits/sec [ 5] 7.00-8.00 sec 1.93 MBytes 16.2 Mbits/sec [ 5] 8.00-9.00 sec 2.26 MBytes 18.9 Mbits/sec [ 5] 9.00-10.00 sec 1.81 MBytes 15.2 Mbits/sec [ 5] 10.00-10.04 sec 0.00 Bytes 0.00 bits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.04 sec 753 MBytes 629 Mbits/sec receiver ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- ^Ciperf3: interrupt - the server has terminated root@Jupiter:~# ^C root@Jupiter:~# iperf3 -s ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 192.168.1.145, port 51737 [ 5] local 192.168.1.105 port 5201 connected to 192.168.1.145 port 51738 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 2.23 MBytes 18.7 Mbits/sec [ 5] 1.00-2.00 sec 1.84 MBytes 15.4 Mbits/sec [ 5] 2.00-3.00 sec 2.95 MBytes 24.7 Mbits/sec [ 5] 3.00-4.00 sec 1.85 MBytes 15.5 Mbits/sec [ 5] 4.00-5.00 sec 2.74 MBytes 23.0 Mbits/sec [ 5] 5.00-6.00 sec 151 MBytes 1.27 Gbits/sec [ 5] 6.00-7.00 sec 318 MBytes 2.67 Gbits/sec [ 5] 7.00-8.00 sec 319 MBytes 2.68 Gbits/sec [ 5] 8.00-9.00 sec 319 MBytes 2.67 Gbits/sec [ 5] 9.00-10.00 sec 316 MBytes 2.65 Gbits/sec [ 5] 10.00-10.04 sec 12.0 MBytes 2.70 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.04 sec 1.41 GBytes 1.21 Gbits/sec receiver ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Quote Link to comment
JorgeB Posted May 18, 2020 Share Posted May 18, 2020 4 minutes ago, aerosmith9110 said: Is this normal? Not really, you should get reasonably consistent results close to line speed. Quote Link to comment
aerosmith9110 Posted May 18, 2020 Share Posted May 18, 2020 ok then something funky going on. I'll do a reboot. Quote Link to comment
aerosmith9110 Posted May 19, 2020 Share Posted May 19, 2020 ok weirdness again: Desktop: C:\iperf3>iperf3 -c 192.168.1.105 Connecting to host 192.168.1.105, port 5201 [ 4] local 192.168.1.149 port 49757 connected to 192.168.1.105 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 256 KBytes 2.10 Mbits/sec [ 4] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec [ 4] 2.00-3.00 sec 47.9 MBytes 402 Mbits/sec [ 4] 3.00-4.00 sec 56.5 MBytes 474 Mbits/sec [ 4] 4.00-5.00 sec 56.0 MBytes 470 Mbits/sec [ 4] 5.00-6.00 sec 56.2 MBytes 472 Mbits/sec [ 4] 6.00-7.00 sec 56.2 MBytes 471 Mbits/sec [ 4] 7.00-8.00 sec 56.1 MBytes 472 Mbits/sec [ 4] 8.00-9.00 sec 56.4 MBytes 473 Mbits/sec [ 4] 9.00-10.00 sec 56.2 MBytes 472 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 442 MBytes 371 Mbits/sec sender [ 4] 0.00-10.00 sec 442 MBytes 371 Mbits/sec Unraid: root@Jupiter:~# iperf3 -c 192.168.1.149 Connecting to host 192.168.1.149, port 5201 [ 4] local 192.168.1.105 port 59908 connected to 192.168.1.149 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 364 MBytes 3.06 Gbits/sec 1 402 KBytes [ 4] 1.00-2.00 sec 359 MBytes 3.01 Gbits/sec 1 368 KBytes [ 4] 2.00-3.00 sec 361 MBytes 3.03 Gbits/sec 0 350 KBytes [ 4] 3.00-4.00 sec 364 MBytes 3.05 Gbits/sec 0 350 KBytes [ 4] 4.00-5.00 sec 365 MBytes 3.06 Gbits/sec 0 350 KBytes [ 4] 5.00-6.00 sec 363 MBytes 3.05 Gbits/sec 0 350 KBytes [ 4] 6.00-7.00 sec 360 MBytes 3.02 Gbits/sec 1 350 KBytes [ 4] 7.00-8.00 sec 363 MBytes 3.05 Gbits/sec 0 350 KBytes [ 4] 8.00-9.00 sec 364 MBytes 3.06 Gbits/sec 0 350 KBytes [ 4] 9.00-10.00 sec 365 MBytes 3.06 Gbits/sec 0 350 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 3.54 GBytes 3.04 Gbits/sec 3 sender [ 4] 0.00-10.00 sec 3.54 GBytes 3.04 Gbits/sec receiver currently I am at 3gbps wall. Do you have any idea why results like that? Quote Link to comment
JorgeB Posted May 19, 2020 Share Posted May 19, 2020 52 minutes ago, aerosmith9110 said: Do you have any idea why results like that? You'd need to try different things, NICs, cables, switch, etc. Quote Link to comment
aerosmith9110 Posted May 30, 2020 Share Posted May 30, 2020 (edited) ok, still figuring things out... if you have something to add . I know changing more hardware might solve the problem but if you look at the test below it shows that the HW can do the correct speed. Thus, if say the card of card on server 1 is bad then why does it not manifest itself on the test server 1 to server 2 something to that effect.... setup: PC connected to sw A Switch A connected to switch B ( LACP ) I tried to disable one of the legs still same issue. Unraid server 1 connected to Switch B ( LACP ) Unraid server 2 connected to Switch B Unraid server ( tx and RX ) = 8 - 9 gbps PC to Server 1 = Not even 1gbps Server 1 to PC = 9gbps PC to Server 2 = 9gbps Server 2 to PC = 9gbps I am still doing some test but I posted here to track my progress rather than putting it on notepad.. at least here someone might benefit or might be able to help explain.. Edited May 30, 2020 by aerosmith9110 Quote Link to comment
aerosmith9110 Posted May 31, 2020 Share Posted May 31, 2020 (edited) Just checked. I can do 10gbps by doing 2 stream to 1 server... Do you know what is giving me the limit? ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 192.168.1.145, port 60061 [ 5] local 192.168.1.105 port 5201 connected to 192.168.1.145 port 60062 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 804 MBytes 6.75 Gbits/sec [ 5] 1.00-2.00 sec 838 MBytes 7.03 Gbits/sec [ 5] 2.00-3.00 sec 838 MBytes 7.03 Gbits/sec [ 5] 3.00-4.00 sec 844 MBytes 7.08 Gbits/sec [ 5] 4.00-5.00 sec 840 MBytes 7.04 Gbits/sec [ 5] 5.00-6.00 sec 843 MBytes 7.07 Gbits/sec [ 5] 6.00-7.00 sec 841 MBytes 7.05 Gbits/sec [ 5] 7.00-8.00 sec 837 MBytes 7.02 Gbits/sec [ 5] 8.00-9.00 sec 840 MBytes 7.05 Gbits/sec [ 5] 9.00-10.00 sec 836 MBytes 7.02 Gbits/sec [ 5] 10.00-10.04 sec 34.6 MBytes 7.51 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.04 sec 8.20 GBytes 7.02 Gbits/sec receiver ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 192.168.1.145, port 60074 [ 5] local 192.168.1.105 port 5201 connected to 192.168.1.145 port 60075 [ 7] local 192.168.1.105 port 5201 connected to 192.168.1.145 port 60076 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 593 MBytes 4.98 Gbits/sec [ 7] 0.00-1.00 sec 537 MBytes 4.50 Gbits/sec [SUM] 0.00-1.00 sec 1.10 GBytes 9.48 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 1.00-2.00 sec 614 MBytes 5.15 Gbits/sec [ 7] 1.00-2.00 sec 552 MBytes 4.63 Gbits/sec [SUM] 1.00-2.00 sec 1.14 GBytes 9.78 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 2.00-3.00 sec 614 MBytes 5.15 Gbits/sec [ 7] 2.00-3.00 sec 556 MBytes 4.67 Gbits/sec [SUM] 2.00-3.00 sec 1.14 GBytes 9.82 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 3.00-4.00 sec 611 MBytes 5.13 Gbits/sec [ 7] 3.00-4.00 sec 564 MBytes 4.73 Gbits/sec [SUM] 3.00-4.00 sec 1.15 GBytes 9.85 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 4.00-5.00 sec 610 MBytes 5.12 Gbits/sec [ 7] 4.00-5.00 sec 564 MBytes 4.74 Gbits/sec [SUM] 4.00-5.00 sec 1.15 GBytes 9.85 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 5.00-6.00 sec 610 MBytes 5.12 Gbits/sec [ 7] 5.00-6.00 sec 565 MBytes 4.74 Gbits/sec [SUM] 5.00-6.00 sec 1.15 GBytes 9.85 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 6.00-7.00 sec 611 MBytes 5.13 Gbits/sec [ 7] 6.00-7.00 sec 565 MBytes 4.74 Gbits/sec [SUM] 6.00-7.00 sec 1.15 GBytes 9.86 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 7.00-8.00 sec 611 MBytes 5.13 Gbits/sec [ 7] 7.00-8.00 sec 563 MBytes 4.73 Gbits/sec [SUM] 7.00-8.00 sec 1.15 GBytes 9.85 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 8.00-9.00 sec 610 MBytes 5.11 Gbits/sec [ 7] 8.00-9.00 sec 565 MBytes 4.74 Gbits/sec [SUM] 8.00-9.00 sec 1.15 GBytes 9.86 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 9.00-10.00 sec 609 MBytes 5.11 Gbits/sec [ 7] 9.00-10.00 sec 567 MBytes 4.75 Gbits/sec [SUM] 9.00-10.00 sec 1.15 GBytes 9.86 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 10.00-10.03 sec 20.4 MBytes 5.08 Gbits/sec [ 7] 10.00-10.03 sec 19.3 MBytes 4.81 Gbits/sec [SUM] 10.00-10.03 sec 39.8 MBytes 9.89 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.03 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.03 sec 5.97 GBytes 5.11 Gbits/sec receiver [ 7] 0.00-10.03 sec 0.00 Bytes 0.00 bits/sec sender [ 7] 0.00-10.03 sec 5.49 GBytes 4.70 Gbits/sec receiver [SUM] 0.00-10.03 sec 0.00 Bytes 0.00 bits/sec sender [SUM] 0.00-10.03 sec 11.5 GBytes 9.81 Gbits/sec receiver ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- BTW PC to Server 1 = Not even 1gbps - This already is getting 7gbps.. not sure what happened. Edited May 31, 2020 by aerosmith9110 Quote Link to comment
aerosmith9110 Posted June 7, 2020 Share Posted June 7, 2020 (edited) On 5/19/2020 at 8:41 PM, johnnie.black said: You'd need to try different things, NICs, cables, switch, etc. Not sure if you would notice my post.. seems that the ram cache is not freeing itself even after leaving it overnight. I got this settings: Current NIC settings Flow Control: On NIC Offload: On NIC Rx Buffer: 256 NIC Tx Buffer: 256 * N/A - The feature is not available on any of your NICs. Current Disk Cache Settings vm.dirty_background_ratio = 1 % vm.dirty_ratio = 90 % Current Inotify Settings fs.inotify.max_user_watches = 524288 CPU Frequency Scaling Driver: Intel Pstate Governor: Performance But I am basing it on my dyna system stats. but I transferred about 1gb and it was 540MB/s which is way beyond my hdds. am I right to assume that the contents of the ram is actually written already and the ram just let the contents stay there for the time beeing until new data is written and ram is freed during new file transfers? Edited June 7, 2020 by aerosmith9110 Quote Link to comment
JorgeB Posted June 8, 2020 Share Posted June 8, 2020 9 hours ago, aerosmith9110 said: am I right to assume that the contents of the ram is actually written already and the ram just let the contents stay there for the time beeing No, they are flushed to disk after a few seconds (or when the RAM cache is full). Quote Link to comment
aerosmith9110 Posted June 10, 2020 Share Posted June 10, 2020 On 6/8/2020 at 4:34 PM, johnnie.black said: No, they are flushed to disk after a few seconds (or when the RAM cache is full). mine does not flush the ram. the ram cache is always full and stays forever until reboot.. so to test my theory I copied several large files and quickly deleted it.. the array stopped writing and the ram suddenly freed up a bunch of space but not all of it ( I would assume the space that I recently copied and deleted was removed from ram )... basically my cache is always full but I am still getting my normal speed before I hit my ram capacity. I am not sure what is happening ... weird since you mentioned it should clear the cache after writing but I have a full cache but I am assuming it is already written on the array but it has a copy in ram which is freed by the system if the ram is needed.. but this is just a theory.... If you are sure that this isn't supposed to happen perhaps you can give me directions to check it out... Quote Link to comment
JorgeB Posted June 10, 2020 Share Posted June 10, 2020 4 minutes ago, aerosmith9110 said: the ram cache is always full and stays forever until reboot.. That's RAM used for read cache, all writes are flushed to disk after a few seconds. Quote Link to comment
aerosmith9110 Posted June 10, 2020 Share Posted June 10, 2020 Just now, johnnie.black said: That's RAM used for read cache, all writes are flushed to disk after a few seconds. understood so basically read cache stays forever then just flushes it when the ram is needed? Quote Link to comment
JorgeB Posted June 10, 2020 Share Posted June 10, 2020 Yes, writes are flushed after a few seconds to protect against for example a power loss, but all available RAM is used for read cache. https://www.linuxatemyram.com/ Quote Link to comment
aerosmith9110 Posted June 10, 2020 Share Posted June 10, 2020 1 minute ago, johnnie.black said: Yes, writes are flushed after a few seconds to protect against for example a power loss, but all available RAM is used for read cache. https://www.linuxatemyram.com/ tnx. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.