opentoe

Still going strong here on RC5. My copying is back to normal. I first posted about this on March 10th this year. I should have read the bug reports more often.

This sounds pretty nice and good information. I have an Unraid server and a few network clients connected through a 1Gb switch. For me to benefit the most with this would I need to switch all my workstations and switch and Unraid box to 10G? Nice looking plugin. What's that one called?

Ok, so it really only benefits users with SSD drives. Ok.

What are people using 10Gb NICs for? If you have an Unraid server with mechanical drives would a 10Gb card even help?

I concur. I had to do this as well.

Well can't always rely on what speeds Windows shows. I also did a GCP copy via console and it worked great. I also checked the tunable settings and it looks like they were reset to the defaults which I'm just leaving alone and never touching again. I did see that FUSE layer setting. Experimental: If set to Yes then mount User Share file system with FUSE direct_io mount option. This will increase write performance but might possibly decrease read performance. Auto selects No Mine is set to AUTO stripes = 4096 queue limit = 80 sync limit = 5 write method = auto

Ok, this is odd. The only thing I have done was upgrade to the beta 6.8.0rc4. So for the hell of it I tried a Windows copy after the server settled down and idled for a good 10-15 minutes. This is the first time I've seen a normal copy from within the array itself in about a year. There was no camels back with all those troughs and spikes. This was a consistent copy that I am used to seeing. So no one thinks I'm crazy I cut a short video of the end of the copy. You'll see a night and day difference between this copy video and the one in the original post. The difference is very apparent. Could this have been the issue jonnie.black was indicating in his post? I sure as hell hope so. Don't want to get all excited here so I'll do some more testing when time permits maybe over this weekend. I also did a console ( MC ) copy and it didn't stall out at all. Just a 'little' happy here. Haven't seen that type of copy in a very long time. Anyway, check out the first video in the original post and then this one. Wow. unraidcopywithbeta.mp4

Here is the diag data. Don't even worry about the Windows video copy. Once we are into the console itself and the copy stays at 0% for 2 minutes we can eliminate the internal network. Not using SMB while doing a console copy, so the network topology should not matter but the network is 1Gb through out, all segments tested while the network was being installed. Here are my tunable parameters: Tunable NR REQUESTS: 1024 Tunable MD_NUM STRIPES: 4096 Tunable MD_SYNC WINDOW: 2048 Tunable MD_SYNC THRESH: 2000 Tunable MD_WRITE_METHOD: RECONSTRUCT WRITE I did run a utility that ran for quite a bit that gave me the tunable recommended numbers. I'll see if I can find that utility again and run the test again. I'll see if that utility works with the latest version. I also wanted to add. I have a PRO LICENSE key that I won during our get together one night for Unraid. I have never used this key 'yet' since I already have another small box running with all SSD's just for a couple small things. Whoever finds out what the issue is the key is yours! I'll get it transferred to you. Maybe will motivate. I'm just burned out trying to figure out what it could be and really been slammed with year end projects at work and rarely have the time to tinker. BTW, happy Holllla Ween! sun-diagnostics-20191101-0208.zip

It was happening before 6.7.2 but I will def try the beta for sure.

Hi, all my Windows machines are using SMBV2/V3. Used the powershell command to find out what version it is. Get-SmbServerConfiguration | Select EnableSMB2Protocol To eliminate all that stuff I consoled into the Unraid box and used MC to perform the copy. That also would come to a complete stop. Here is a video of that using the built in MC, which I do use a lot. I would usually see that blue bar move steadily at 55-60 MB/sec. Something made a change a while back in V6 that somehow slowed my copies big time. Here is a short video that shows how slow and bad it really is. Not being able to move files around easily hinders using Unraid big time. Very weird, right? unraidcopymc.mp4

Can someone try to do a copy with a large file from one user share to another and post the results up here? I did previously post about this maybe couple months ago but wanted a fresh look and maybe new eyes that can possibly help resolve this PIA issue. Copying a single 40 GB file takes about 15 minutes. Anyway I made another small vid of a Windows copy showing what happens and also a small vid of the basic resources page while the copy is happening. I just want to say when I copied on previous version of unraid it would keep a consistent speed. Version 5 was solid too. I remember doing A LOT of copying when going from BFRS to XFS. If I had the space to move all the content I would reformat the entire system. If I had the big bucks I would remove all the drives in the system and install all new drives for testing. Have to add I have replaced everything in the system except the processor. Mainboard, JBOD cards, memory, cables, formatted my USB boot drive. Also I always leave all drives spun up 24/7. Any suggestions are welcome. unraidcopy.mp4 resources.mp4

Currently have an Intel i7-3930k processor which is still fast and powerful enough to run a nice unraid box. I also have 8 DDR3 chips adding up to 32GB of memory. So instead of replacing the processor and memory, I just want to replace the mainboard. The socket would be LGA-2011. I really would like to try out a Gigabyte model since I have never owned one. I always stuck with Asus but am ready for a change. Anyone recommend a good LGA-2011 Gigabye board with at least 8 memory slots ( can be 4 if it has to be ) and 3 PCI-E slots? I see so many "gaming" boards out there, but I'm also looking for virtualization capabilities. Been on Ebay since the LGA-2011 socket isn't made anymore and most boards are pre-owned with those sockets.

I found an old post that does confirm my copy speeds within the array were at one point around 100MB/sec with construct on. This was when Unraid just turned to V6. I read the whole thread and settles me that my system, which I inherited from my friend is now my Norco server I'm using now. Seeing that my speeds ended up at 60MB/sec for constant and consistent speed was good to read, so it has to be "something". I'm sure I can even find more posts I made years ago about copying speeds but I'm not very good at filtering them or finding them. The post below is from 3 years ago. I'm positive there are more out there showing or me saying what my copying speeds were/are. It is just finding all posts by user "opentoe" and then keyword search on those posts. Not sure where to find that. I do see options when I click on the search box, but wanted a little more and could not find it. Would just like to see at least 1 more post from years ago that my copy speeds were good. The ONLY possible thing that I have not tried was new back planes for the Norco case. They all have solid state logic boards on them, so it could be possible there are issues with them? If they are cheap I will what it cost to replace those. Always going to bed with a headache at night.

My Malware protection keeps popping up when browsing this forum. Just want to make sure you guys are using Google-Analytics so I can bypass the errors and stop the pop-ups. To confirm, are you using Google-Analytics?

I never use disk shares. I was only testing the array to see if it would make any difference on my array to array copy speeds. Any of V5 releases I got exceptional copy speeds from the array to array with it never turning into a camels back with all the dips. I will test the same scenario with RECONSTRUCT OFF and see if that helps. I'll Google search the terms and see what's there. Thanks. UPDATE: Took RECONSTRUCT off and things just got worse. Doesn't read from all the disks at all, but the copy is so random with the speeds it is really frustrating. Ever since V6 these things started to happen. I don't use a cache drive but was always happy with a constant 60MB/sec of copying anything from array to array all the time. I sort of miss that and just can't explain or think what could of happened. All my drives are 7200 RPM with at least 128MB cache, not to say that's going to make it fly but this same exact hardware with using V5 had no problems at all. I think even V3 I had and used was really good. Started with Unraid from the start....years ago. I've replaced ALL hardware except the drives themselves. Just did a parity check which took 18 hours and the average speed was 122MB/sec. That's not bad. I've also done extensive tests on the individual drives themselves, all came back with good results. The only option at this point is to build on my Synology which I do not want to do or build a parallel system with Unraid and throw some data on it and see what happens. But I can't use it the way it is now. Copy drops to 0MB/sec, that shouldn't happen. Anyway, if curious here is what the copy looks like with RECONSTRUCT at Auto. http://www.diggsnet.com/unraidfilecopy3.mp4 BTW, I never spin down my drives. They are always spinning 24/7/365.

Tunable (md_write_method): RECONSTRUCT WRITE Been on that setting the whole time while I try to get my camel back copies back to normal. Not that this "should not" matter, but all my user shares are setup as: Allocation method: MOST-FREE Min Free Space: 25GB Split Level: Automatically split any directory as required So no ideas on why it would be reading from ALL disks when the file itself it physically on one drive ( drive 9 ) that is. If any of the devs or Unraid mods that really know the insides of Unraid why is it trying to read from ALL my disks? I think that is what is killing my file copies. So what I did was exported all my drives individually. The file is on disk 9, so I used the disk share and copied the file from disk 9 to just another user share and the results were the video below. I also toggled the READ/WRITES display to show the throughput and you'll see Unraid is reading from ALL my drives when I'm just trying to copy a file from drive 9 to a user share. This could be a start of finally seeing the root of this issue that has plagued me for a long time now. Here is the video, you would be surprised to see all my drives show reads. Does not look right. http://www.diggsnet.com/unraidfilecopy2.mp4

I've noticed that when copying a large file it is reading from ALL the drives, which I believe is what is slowing down my copies which have been going on for several months now. I still can't get a clean copy straight though. It dips, goes back up then dips down again. I did record another video and if you notice ALL my drives change as they are all being read from, which is kind of weird, no? Does Unraid save large files in pieces on multiple drives across the array? Just wondering why it is reading from ALL drives when copying one file. I know there are settings on how Unraid should store files. Anyway, here is the small video I made, as you can see it looks like a camel's back with all the humps. And yea, it still drops down to 0 MB/sec and takes forever to copy any amount of files. I did give up on Unraid a little, been using it since I believe V3 and trying different solutions, but still want to keep my Unraid box running. Here's a link to the video. http://www.diggsnet.com/filecopy.mp4 Just really wondering why it is reading from ALL disks on a one file copy with a large file. Unless that file is spread out on multiple disks? You'll notice all the drives increase their READ stats all at the same time. Also when copying that same file from the array to a drive outside the array it is a steady flatline all the way across 80MB/sec. Doesn't dip at all.

Copy a large file from your array or a folder with lots of files (20 GB of data should be fine) and see what speeds you are getting. If you are receiving slow speeds during the copy or dips down to 0 like I do, maybe you have my plague. Worth a check and simple to test.

I got it installed using the directions in the first post. It worked ok. All test speeds were right on mark. Thanks.

Has the normal installation of this been fixed or you still need to manually install?

Thanks. That worked fine. I completely forgot there were options for UD. I started from scratch on all of them and reformatted and created new partitions and mounted. They show up in a folder /mnt/disks/<unassigned disks> but I guess that's normal. Going to use them for some testing/etc. Will be almost humorous to see files being copied with SSD drives come to a complete stop. Thank you.

Trying to format and mount using unassigned devices. I did make an XFS file system and mounted "unassigned1" to it through the command line but usually this easily happens through the GUI. How would I basically start from scratch on disk 1 and mount "unassigned1" to it upon bootup through unassigned devices. Here is a screen shot on what it looks like. When I click mount the link is dead. These are new drives and I do not care if they get wiped or dumped. Using them for side storage right now.

I tried to do what you said, still errored out.

When I performed the terminal copy test using midnight commander I used disk shares and made sure the source disk was not on the destination controller. So there shouldn't have been any issues with one 6GB file. Like I said previously I never had any issues like this. I actually had more disks in the array and three controllers. Thinking it would help by removing a couple of drives and also removing a third controller didn't have any affect at all. I use to get 112MB/sec on large copies, weather or not the files were small or large. I could be copying/moving 2-3 terabytes and consistently get 112MB/sec. I remember this because there was a bug push a while ago when a lot of us were changing the file system from btrfs to xfs and that caused A LOT of file copying. Some copies lasted hours and hours and would never dip down below 100MB/sec during the copy. I have a lot of memory ( 32GB ) for creating some VM's and running dockers. Because of this issue and I'm not sure where it is going to take me I removed all my dockers and removed all my VM's. You'd thik when copying a 6GB file there wouldn't be any caching going on since it could fit the whole file in memory. I don't know if any other users notice issues like this, but when you have been running unraid for such a long time you notice every little thing. I thought I would have this issue settled in a few days or less but since it is going on for several months not much I can do. And I do get fantastic speeds during a parity sync. No dipping in speeds at all, as one time I almost watched the entire check just to make sure, all because I really wanted to get to the bottom of this. It is pretty sad. I deleted weeks and weeks worth of DVD's and movies that I have ripped knowing I can rip them back but not touching one disk until all problems are fixed and box runs like it use to.

Prior to having that option in the disk settings I use to have it in my "go" file. I believe it was md_write_method 1. Yea, I've went back and forth with trying AUTO/READ-WRITE-MODIFY/RECONSTRUCT-WRITE with no improvement. I've kept it on RECONSTRUCT.