opentoe Posted October 27, 2019 Share Posted October 27, 2019 Can someone try to do a copy with a large file from one user share to another and post the results up here? I did previously post about this maybe couple months ago but wanted a fresh look and maybe new eyes that can possibly help resolve this PIA issue. Copying a single 40 GB file takes about 15 minutes. Anyway I made another small vid of a Windows copy showing what happens and also a small vid of the basic resources page while the copy is happening. I just want to say when I copied on previous version of unraid it would keep a consistent speed. Version 5 was solid too. I remember doing A LOT of copying when going from BFRS to XFS. If I had the space to move all the content I would reformat the entire system. If I had the big bucks I would remove all the drives in the system and install all new drives for testing. Have to add I have replaced everything in the system except the processor. Mainboard, JBOD cards, memory, cables, formatted my USB boot drive. Also I always leave all drives spun up 24/7. Any suggestions are welcome. unraidcopy.mp4 resources.mp4 Quote Link to comment
Squid Posted October 28, 2019 Share Posted October 28, 2019 Looks like its caching and the pauses are due to it writing it out. Few questions though. Is unRaid and Windows both using SMB3? (SMB3 is tons faster because the data doesn't have to traverse the network if source and destination are the same mount point.) Similarily, are you copying from U: to U: ? Quote Link to comment
opentoe Posted October 31, 2019 Author Share Posted October 31, 2019 On 10/27/2019 at 11:13 PM, Squid said: Looks like its caching and the pauses are due to it writing it out. Few questions though. Is unRaid and Windows both using SMB3? (SMB3 is tons faster because the data doesn't have to traverse the network if source and destination are the same mount point.) Similarily, are you copying from U: to U: ? Hi, all my Windows machines are using SMBV2/V3. Used the powershell command to find out what version it is. Get-SmbServerConfiguration | Select EnableSMB2Protocol To eliminate all that stuff I consoled into the Unraid box and used MC to perform the copy. That also would come to a complete stop. Here is a video of that using the built in MC, which I do use a lot. I would usually see that blue bar move steadily at 55-60 MB/sec. Something made a change a while back in V6 that somehow slowed my copies big time. Here is a short video that shows how slow and bad it really is. Not being able to move files around easily hinders using Unraid big time. Very weird, right? unraidcopymc.mp4 Quote Link to comment
JorgeB Posted October 31, 2019 Share Posted October 31, 2019 There's a known issue with v6.7.2 where any array writes starve reads, this also affects moving data from one array device to another, it was fixed on v6.8-rc, you can try that. Quote Link to comment
Vr2Io Posted October 31, 2019 Share Posted October 31, 2019 Pls post diagnostic file. During SMB network transfer, it shoot over 110MB/s, does network in 10G or bonding ? No matter in 6.7 or 6.8, disk tunable parameter need change, otherwise performance will reduce a lot. Direct I/O and turbo-write setting also affect too. Pls try evaluates those setting. Quote Link to comment
opentoe Posted November 1, 2019 Author Share Posted November 1, 2019 17 hours ago, johnnie.black said: There's a known issue with v6.7.2 where any array writes starve reads, this also affects moving data from one array device to another, it was fixed on v6.8-rc, you can try that. It was happening before 6.7.2 but I will def try the beta for sure. Quote Link to comment
opentoe Posted November 1, 2019 Author Share Posted November 1, 2019 (edited) 18 hours ago, Benson said: Pls post diagnostic file. During SMB network transfer, it shoot over 110MB/s, does network in 10G or bonding ? No matter in 6.7 or 6.8, disk tunable parameter need change, otherwise performance will reduce a lot. Direct I/O and turbo-write setting also affect too. Pls try evaluates those setting. Here is the diag data. Don't even worry about the Windows video copy. Once we are into the console itself and the copy stays at 0% for 2 minutes we can eliminate the internal network. Not using SMB while doing a console copy, so the network topology should not matter but the network is 1Gb through out, all segments tested while the network was being installed. Here are my tunable parameters: Tunable NR REQUESTS: 1024 Tunable MD_NUM STRIPES: 4096 Tunable MD_SYNC WINDOW: 2048 Tunable MD_SYNC THRESH: 2000 Tunable MD_WRITE_METHOD: RECONSTRUCT WRITE I did run a utility that ran for quite a bit that gave me the tunable recommended numbers. I'll see if I can find that utility again and run the test again. I'll see if that utility works with the latest version. I also wanted to add. I have a PRO LICENSE key that I won during our get together one night for Unraid. I have never used this key 'yet' since I already have another small box running with all SSD's just for a couple small things. Whoever finds out what the issue is the key is yours! I'll get it transferred to you. Maybe will motivate. I'm just burned out trying to figure out what it could be and really been slammed with year end projects at work and rarely have the time to tinker. BTW, happy Holllla Ween! sun-diagnostics-20191101-0208.zip Edited November 1, 2019 by opentoe Change Quote Link to comment
opentoe Posted November 1, 2019 Author Share Posted November 1, 2019 Ok, this is odd. The only thing I have done was upgrade to the beta 6.8.0rc4. So for the hell of it I tried a Windows copy after the server settled down and idled for a good 10-15 minutes. This is the first time I've seen a normal copy from within the array itself in about a year. There was no camels back with all those troughs and spikes. This was a consistent copy that I am used to seeing. So no one thinks I'm crazy I cut a short video of the end of the copy. You'll see a night and day difference between this copy video and the one in the original post. The difference is very apparent. Could this have been the issue jonnie.black was indicating in his post? I sure as hell hope so. Don't want to get all excited here so I'll do some more testing when time permits maybe over this weekend. I also did a console ( MC ) copy and it didn't stall out at all. Just a 'little' happy here. Haven't seen that type of copy in a very long time. Anyway, check out the first video in the original post and then this one. Wow. unraidcopywithbeta.mp4 Quote Link to comment
Vr2Io Posted November 1, 2019 Share Posted November 1, 2019 (edited) 6.8 have major change in FUSE layer ( assume test on array, not cache pool or UD ), the behavior also different. BTW, tuning on the new set of tunable could improve an make things more smooth. 77.4MB/s should be normal performance. I quick check on the diagnostic, in general no abnormal found. 41 minutes ago, opentoe said: but the network is 1Gb Strange, I never foud an overshoot > 113 MB/s when doing SMB transfer on 1Gbps ethernet. Below are some practical transfer in Unraid view, most time in saturate but some deeps not avoidable. Edited November 1, 2019 by Benson Quote Link to comment
opentoe Posted November 1, 2019 Author Share Posted November 1, 2019 Well can't always rely on what speeds Windows shows. I also did a GCP copy via console and it worked great. I also checked the tunable settings and it looks like they were reset to the defaults which I'm just leaving alone and never touching again. I did see that FUSE layer setting. Experimental: If set to Yes then mount User Share file system with FUSE direct_io mount option. This will increase write performance but might possibly decrease read performance. Auto selects No Mine is set to AUTO stripes = 4096 queue limit = 80 sync limit = 5 write method = auto Quote Link to comment
Vr2Io Posted November 1, 2019 Share Posted November 1, 2019 (edited) Yes, different setting have different behaviour, just tune and got what's you like. Edited November 1, 2019 by Benson Quote Link to comment
Squid Posted November 1, 2019 Share Posted November 1, 2019 57 minutes ago, Benson said: Strange, I never foud an overshoot > 113 MB/s when doing SMB transfer on 1Gbps ethernet. It will if the source and destination are on the server. In that case, the data never actually traverses the network with SMB3 Quote Link to comment
Vr2Io Posted November 1, 2019 Share Posted November 1, 2019 15 minutes ago, Squid said: It will if the source and destination are on the server. In that case, the data never actually traverses the network with SMB3 Yes, if it is the case. From previous video, I notice a rather long slow/pause period during network transfer, so I assume actual file in transfer there. Quote Link to comment
Vr2Io Posted November 1, 2019 Share Posted November 1, 2019 59 minutes ago, opentoe said: write method = auto auto means turbo write off, if you enjoy speed, you should set to on. Those affect on write to array only. Quote Link to comment
opentoe Posted November 6, 2019 Author Share Posted November 6, 2019 Still going strong here on RC5. My copying is back to normal. I first posted about this on March 10th this year. I should have read the bug reports more often. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.