Jump to content

opentoe

Members
  • Content Count

    1726
  • Joined

  • Last visited

Community Reputation

7 Neutral

About opentoe

  • Rank
    Advanced Member

Converted

  • Gender
    Undisclosed

Recent Profile Visitors

906 profile views
  1. I found an old post that does confirm my copy speeds within the array were at one point around 100MB/sec with construct on. This was when Unraid just turned to V6. I read the whole thread and settles me that my system, which I inherited from my friend is now my Norco server I'm using now. Seeing that my speeds ended up at 60MB/sec for constant and consistent speed was good to read, so it has to be "something". I'm sure I can even find more posts I made years ago about copying speeds but I'm not very good at filtering them or finding them. The post below is from 3 years ago. I'm positive there are more out there showing or me saying what my copying speeds were/are. It is just finding all posts by user "opentoe" and then keyword search on those posts. Not sure where to find that. I do see options when I click on the search box, but wanted a little more and could not find it. Would just like to see at least 1 more post from years ago that my copy speeds were good. The ONLY possible thing that I have not tried was new back planes for the Norco case. They all have solid state logic boards on them, so it could be possible there are issues with them? If they are cheap I will what it cost to replace those. Always going to bed with a headache at night.
  2. My Malware protection keeps popping up when browsing this forum. Just want to make sure you guys are using Google-Analytics so I can bypass the errors and stop the pop-ups. To confirm, are you using Google-Analytics?
  3. I never use disk shares. I was only testing the array to see if it would make any difference on my array to array copy speeds. Any of V5 releases I got exceptional copy speeds from the array to array with it never turning into a camels back with all the dips. I will test the same scenario with RECONSTRUCT OFF and see if that helps. I'll Google search the terms and see what's there. Thanks. UPDATE: Took RECONSTRUCT off and things just got worse. Doesn't read from all the disks at all, but the copy is so random with the speeds it is really frustrating. Ever since V6 these things started to happen. I don't use a cache drive but was always happy with a constant 60MB/sec of copying anything from array to array all the time. I sort of miss that and just can't explain or think what could of happened. All my drives are 7200 RPM with at least 128MB cache, not to say that's going to make it fly but this same exact hardware with using V5 had no problems at all. I think even V3 I had and used was really good. Started with Unraid from the start....years ago. I've replaced ALL hardware except the drives themselves. Just did a parity check which took 18 hours and the average speed was 122MB/sec. That's not bad. I've also done extensive tests on the individual drives themselves, all came back with good results. The only option at this point is to build on my Synology which I do not want to do or build a parallel system with Unraid and throw some data on it and see what happens. But I can't use it the way it is now. Copy drops to 0MB/sec, that shouldn't happen. Anyway, if curious here is what the copy looks like with RECONSTRUCT at Auto. http://www.diggsnet.com/unraidfilecopy3.mp4 BTW, I never spin down my drives. They are always spinning 24/7/365.
  4. Tunable (md_write_method): RECONSTRUCT WRITE Been on that setting the whole time while I try to get my camel back copies back to normal. Not that this "should not" matter, but all my user shares are setup as: Allocation method: MOST-FREE Min Free Space: 25GB Split Level: Automatically split any directory as required So no ideas on why it would be reading from ALL disks when the file itself it physically on one drive ( drive 9 ) that is. If any of the devs or Unraid mods that really know the insides of Unraid why is it trying to read from ALL my disks? I think that is what is killing my file copies. So what I did was exported all my drives individually. The file is on disk 9, so I used the disk share and copied the file from disk 9 to just another user share and the results were the video below. I also toggled the READ/WRITES display to show the throughput and you'll see Unraid is reading from ALL my drives when I'm just trying to copy a file from drive 9 to a user share. This could be a start of finally seeing the root of this issue that has plagued me for a long time now. Here is the video, you would be surprised to see all my drives show reads. Does not look right. http://www.diggsnet.com/unraidfilecopy2.mp4
  5. I've noticed that when copying a large file it is reading from ALL the drives, which I believe is what is slowing down my copies which have been going on for several months now. I still can't get a clean copy straight though. It dips, goes back up then dips down again. I did record another video and if you notice ALL my drives change as they are all being read from, which is kind of weird, no? Does Unraid save large files in pieces on multiple drives across the array? Just wondering why it is reading from ALL drives when copying one file. I know there are settings on how Unraid should store files. Anyway, here is the small video I made, as you can see it looks like a camel's back with all the humps. And yea, it still drops down to 0 MB/sec and takes forever to copy any amount of files. I did give up on Unraid a little, been using it since I believe V3 and trying different solutions, but still want to keep my Unraid box running. Here's a link to the video. http://www.diggsnet.com/filecopy.mp4 Just really wondering why it is reading from ALL disks on a one file copy with a large file. Unless that file is spread out on multiple disks? You'll notice all the drives increase their READ stats all at the same time. Also when copying that same file from the array to a drive outside the array it is a steady flatline all the way across 80MB/sec. Doesn't dip at all.
  6. Copy a large file from your array or a folder with lots of files (20 GB of data should be fine) and see what speeds you are getting. If you are receiving slow speeds during the copy or dips down to 0 like I do, maybe you have my plague. Worth a check and simple to test.
  7. I got it installed using the directions in the first post. It worked ok. All test speeds were right on mark. Thanks.
  8. Has the normal installation of this been fixed or you still need to manually install?
  9. Thanks. That worked fine. I completely forgot there were options for UD. I started from scratch on all of them and reformatted and created new partitions and mounted. They show up in a folder /mnt/disks/<unassigned disks> but I guess that's normal. Going to use them for some testing/etc. Will be almost humorous to see files being copied with SSD drives come to a complete stop. Thank you.
  10. Trying to format and mount using unassigned devices. I did make an XFS file system and mounted "unassigned1" to it through the command line but usually this easily happens through the GUI. How would I basically start from scratch on disk 1 and mount "unassigned1" to it upon bootup through unassigned devices. Here is a screen shot on what it looks like. When I click mount the link is dead. These are new drives and I do not care if they get wiped or dumped. Using them for side storage right now.
  11. I tried to do what you said, still errored out.
  12. When I performed the terminal copy test using midnight commander I used disk shares and made sure the source disk was not on the destination controller. So there shouldn't have been any issues with one 6GB file. Like I said previously I never had any issues like this. I actually had more disks in the array and three controllers. Thinking it would help by removing a couple of drives and also removing a third controller didn't have any affect at all. I use to get 112MB/sec on large copies, weather or not the files were small or large. I could be copying/moving 2-3 terabytes and consistently get 112MB/sec. I remember this because there was a bug push a while ago when a lot of us were changing the file system from btrfs to xfs and that caused A LOT of file copying. Some copies lasted hours and hours and would never dip down below 100MB/sec during the copy. I have a lot of memory ( 32GB ) for creating some VM's and running dockers. Because of this issue and I'm not sure where it is going to take me I removed all my dockers and removed all my VM's. You'd thik when copying a 6GB file there wouldn't be any caching going on since it could fit the whole file in memory. I don't know if any other users notice issues like this, but when you have been running unraid for such a long time you notice every little thing. I thought I would have this issue settled in a few days or less but since it is going on for several months not much I can do. And I do get fantastic speeds during a parity sync. No dipping in speeds at all, as one time I almost watched the entire check just to make sure, all because I really wanted to get to the bottom of this. It is pretty sad. I deleted weeks and weeks worth of DVD's and movies that I have ripped knowing I can rip them back but not touching one disk until all problems are fixed and box runs like it use to.
  13. Prior to having that option in the disk settings I use to have it in my "go" file. I believe it was md_write_method 1. Yea, I've went back and forth with trying AUTO/READ-WRITE-MODIFY/RECONSTRUCT-WRITE with no improvement. I've kept it on RECONSTRUCT.
  14. I do believe there is an inherent bug somewhere that possibly does not like my hardware. I've been using unraid since the beginning of V5 or even before that. When V6 first came out things were good. After a couple of revisions is when these copying issues started. I always would get 100MB/sec on copy speeds using a Windows workstation on the network and even faster using the terminal and midnight commander. My server box is a simple norco 4224 box. I have a 1000 watt power supply which has been replaced. The cables that go from the backplane to the controller cards have been replaced with new ones. I've changed the memory out temporarily with a friend's memory. I've disabled the on-board NIC and purchased an Intel add on card. I've re-seated the processor and ran extensive tests on the processor itself. The mainboard is an Asus X79 that has tried different bioses. I've tried different controllers. Dell PERC 310's and/or the SuperMicro SAS2LP-MV8. I have like 4 of each of these cards. Tried them together and in different combinations. If I did not have so much data, I'd try to go back to V5 and see what performance I would get. Since the copying of files issues happens via the terminal this eliminates pretty much anything outside the server box. I've even taken the mainboard out of the server box and installed it in a small desktop case and threw Windows 10 on it. Problem free. And while I had Windows 10 on it, I ran several heavy duty burn-in test programs just for the heck of it. Flawless. Reinstalled the board back into the 4224 box, booted up just fine. I've also moved my pro key to a new USB flash stick that was vanilla. No plugins installed, plain vanilla. Tried changing some bios settings, but really not that many. I basically turned off all VM related stuff and tried different settings with that. In a nutshell when copying files the system would start to copy just fine around 80-100MB/sec. Then eventually slowly drop down to 0MB/sec and the copy would literally stop completely for about 30-40 seconds. So if I'm copying a large amount of data it could take days to do this. ( Yes, the parity drive has been replaced ) . Takes about 2.5 minutes to copy a 6GB file when it would take seconds before. I don't even attempt to copy amounts of data anymore and I also think this issue is affecting my system as a whole. Specially when watching a movie. During the entire movie it would freeze ( which is where it stops ) then picks up again and the movie catches up again with the player. This happens with my Dune players AND my Roku players. Another oddity I did a parity sync which took 2 hours ( great! ) but the average speed indicates 1.1GB/sec. Is that even possible? Attached a screen shot of that and diagnostics. I'm pretty much lost from here. I did attach a terminal copy that looks very weird, but you'll get the gist of what is happening once you see it. It is just one 6GB file being copied. Any suggestions or recommendations is highly appreciated. Thanks 6gb copy.mp4 sun-diagnostics-20190609-0516.zip
  15. Amazing, just found this thread and the same EXACT issue is happening. Were you able to resolve your issue? During a file copy using Windows or within the terminal window itself the copy would start off great and then just drop to 0 for abot 10-30 seconds then just start up again. I've replaced just about all the hardware except the hard drives themselves and the breakout cables. I also made sure when I was moving files I used drive shares using different controllers. I even had dual parity and though that could be an issue, so i went back to a single parity with the same results. My parity speeds are around 125MB/sec and do not find errors. I have also ran extended tests on all my drives. So heck, maybe you figured it out? Here are my tunable settings. nr_requests 8 md_num_stripes 4096 md_sync_window 2048 md_sync_thresh 2000 md_write_method using just reconstruct write