SudoPacman Posted March 4, 2019 Share Posted March 4, 2019 Hi all, So, I've tried the first transfer of any significant to my new unRAID box... I noticed it was going very slow so went searching and found some posts recommending the reconstructed write config, so that's what I enabled for this transfer (I have since turned it off so my drives can spin down). The copy is from a Windows Server host, off a ISCSI connected Drobo Pro. Admittedly Drobo Pro's are not the fastest, but certainly a lot LOT faster than this! I'll do a test copy off it to a local drive a bit later, thinking about it. Anyway, here is the results from doing a "robocopy /z /e SOURCE TARGET": Total Copied Skipped Mismatch FAILED Extras Dirs : 9 4 5 0 0 0 Files : 472 317 155 0 0 0 Bytes : 152.852 g 98.782 g 54.070 g 0 0 0 Times : 19:41:51 19:41:49 0:00:00 0:00:01 Speed : 1495801 Bytes/sec. Speed : 85.590 MegaBytes/min. I was hoping for around 40 MegaBytes/sec, since that's what it sounds like others seem to get. What's up? I have attached my slightly doctored diagnostics, since don't want to domain name published to all especially. There's another discussion I need to have wrt permissions when unRAID is in a domain, where none of the domain users are available for limiting samba shares to, and cannot create users with the same name, but I'll create another post for that when I have time. Appreciate any help. Cheers! monty-diagnostics-20190304-0921.zip Link to comment
trurl Posted March 4, 2019 Share Posted March 4, 2019 2 hours ago, SudoPacman said: I have attached my slightly doctored diagnostics, since don't want to domain name published to all especially. If you think the anonymized diagnostics aren't anonymized enough please let us know what else is needed. You shouldn't have to "doctor" your diagnostics, and we don't want you to since you might remove information important to solving your problem You have too many variables in your speed test to know where the problem may lie. But... You have a lot of reallocated sectors on disk1. You should be seeing SMART warning indicators on the Dashboard for that disk. I would probably replace it. Possibly that disk is the cause of any speed issue you may have, since disk1 is going to be the first disk chosen when writing to user shares on a new system. Did you test any of your disks before using them? EVERY BIT of Parity PLUS ALL other disks must be reliably read in order to reliably reconstuct a failed disk. Do you have Notifications setup to alert you by email or other agent when Unraid detects a disk problem like this or some other problem? You need to stay on top of these things so they don't accumulate. One problem is often easy to recover from, but if you ignore it until you have multiple problems, maybe not. Link to comment
SudoPacman Posted March 4, 2019 Author Share Posted March 4, 2019 All I did was change the name of my domain. Yes, I have seen some SMART warning errors, and yes, I intend on replacing the disk. I thought the idea of SMART was so that the bad sectors are known and skipped... I'll use unBALANCE to move everything off it later then, and try again. I did not find the Preclear plugin until after I'd built my server. The 4Tb parity drive is brand new; none of the other drives are mind. I'll update once I now it's writing to a drive that's okay. Can I assume from what you're saying that 85 Mb/minute is not typical then? Cheers Link to comment
trurl Posted March 4, 2019 Share Posted March 4, 2019 3 minutes ago, SudoPacman said: I thought the idea of SMART was so that the bad sectors are known and skipped... The reallocated sectors are not used. But it is an indication that the disk has had problems. 7 minutes ago, SudoPacman said: I intend on replacing the disk. 5 minutes ago, SudoPacman said: I'll use unBALANCE to move everything off it later then, and try again. You don't need to move the data off the disk to replace it. The whole point of parity is to rebuild the data onto a replacement disk. If you mean make the disk empty and try the test again, it would probably result in writing to that disk again unless you excluded it from the user share. 8 minutes ago, SudoPacman said: All I did was change the name of my domain. When you made the diagnostics, did you tell it to anonymize them? Where exactly does your "domain" appear in the diagnostics? Link to comment
SudoPacman Posted March 4, 2019 Author Share Posted March 4, 2019 Well yeah, I thought I'd move the files off and drop the disk from the array. That doesn't appear to be that simple though, since setting it to unassigned does not remove it as such, it marks it as missing. Yes, I told it to anonymise them. The domain name was listed here: Search "NAME_REMOVED" (6 hits in 2 files) C:\Temp\monty-diagnostics-20190304-0921\config\ident.cfg (3 hits) Line 7: DOMAIN="NAME_REMOVED.net." Line 8: DOMAIN_SHORT="NAME_REMOVED" Line 24: LOCAL_TLD="NAME_REMOVED.net" C:\Temp\monty-diagnostics-20190304-0921\system\vars.txt (3 hits) Line 1934: [DOMAIN] => NAME_REMOVED.net. Line 1935: [DOMAIN_SHORT] => NAME_REMOVED Line 1951: [LOCAL_TLD] => NAME_REMOVED.net Cheers Link to comment
Niklas Posted March 4, 2019 Share Posted March 4, 2019 If you change something regarding your shares, this will be added to the syslog: Quote Mar 4 15:46:41 Server emhttpd: req (38): shareNameOrig=domains&shareName=domains&shareComment=Saved+VM+instances&shareAllocator=highwater&shareFloor=0&shareSplitLevel=1&shareInclude=&shareExclude=&shareUseCache=only&cmdEditShare=Apply&csrf_token=**************** Mar 4 15:46:41 Server emhttpd: Starting services... Mar 4 15:46:41 Server emhttpd: shcmd (563): /etc/rc.d/rc.samba restart Both share name and share comment visible in the exported syslog. I used domains here but the same applies to all user shares. That does not get anonymized. I don't know if this happens with other changes that shows in the syslog. Using debug logging for some stuff could show some personal information too. Mover logging will show private filenames and so on. Link to comment
SudoPacman Posted March 4, 2019 Author Share Posted March 4, 2019 Ok, I have done some more testing. I added a 1.5Tb drive to the system, formatted it as xfs, but left it in unassigned devices and shared it. The new drive is showing no SMART errors. I'll replace the 2Tb drive with this one in the next few days. I'd still like to know if my performance is typical though, since seems very slow! Here's my testing tonight, with a single 197Mb unRAID flash backup, to and from various destinations using Windows robocopy /z /e: From Drobo to Desktop PCIe (2.0) SSD: ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 1 1 0 0 0 0 Files : 1 1 0 0 0 0 Bytes : 197.35 m 197.35 m 0 0 0 0 Times : 0:00:03 0:00:03 0:00:00 0:00:00 Speed : 54473986 Bytes/sec. Speed : 3117.026 MegaBytes/min. Ended : 04 March 2019 23:41:54 From Drobo to Desktop SATA2 SSD: ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 1 1 0 0 0 0 Files : 1 1 0 0 0 0 Bytes : 197.35 m 197.35 m 0 0 0 0 Times : 0:00:06 0:00:06 0:00:00 0:00:00 Speed : 30608885 Bytes/sec. Speed : 1751.454 MegaBytes/min. Ended : 04 March 2019 19:50:04 From Drobo to Desktop HDD (1.5Tb, same model as below): ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 1 1 0 0 0 0 Files : 1 1 0 0 0 0 Bytes : 197.35 m 197.35 m 0 0 0 0 Times : 0:00:43 0:00:43 0:00:00 0:00:00 Speed : 4773636 Bytes/sec. Speed : 273.149 MegaBytes/min. Ended : 04 March 2019 23:36:34 From Desktop SSD to unRAID (Unassigned Devices 1.5Tb drive mount - no SMART errors): ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 1 0 1 0 0 0 Files : 1 1 0 0 0 1 Bytes : 197.35 m 197.35 m 0 0 0 0 Times : 0:01:07 0:01:07 0:00:00 0:00:00 Speed : 3073161 Bytes/sec. Speed : 175.847 MegaBytes/min. Ended : 04 March 2019 20:09:50 From Drobo to unRAID (Unassigned Devices 1.5Tb drive mount - no SMART errors): ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 1 0 1 0 0 0 Files : 1 1 0 0 0 0 Bytes : 197.35 m 197.35 m 0 0 0 0 Times : 0:01:06 0:01:06 0:00:00 0:00:00 Speed : 3095594 Bytes/sec. Speed : 177.131 MegaBytes/min. Ended : Mon Mar 04 20:21:53 2019 From Drobo to unRAID array (read/modify/write): ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 1 1 0 0 0 0 Files : 1 1 0 0 0 0 Bytes : 197.35 m 197.35 m 0 0 0 0 Times : 0:03:05 0:03:05 0:00:00 0:00:00 Speed : 1115272 Bytes/sec. Speed : 63.816 MegaBytes/min. Ended : Mon Mar 04 20:16:23 2019 From Drobo to unRAID array (reconstruct write): ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 1 1 0 0 0 0 Files : 1 1 0 0 0 0 Bytes : 197.35 m 197.35 m 0 0 0 0 Times : 0:01:20 0:01:20 0:00:00 0:00:00 Speed : 2558909 Bytes/sec. Speed : 146.421 MegaBytes/min. Ended : Mon Mar 04 20:18:49 2019 As you can see, there is no issue reading from the Drobo Pro across the network, since got 3117Mb/s in a best case scenario! Even copying to a physical hard drive still netted me 273Mb/s - would be pretty happy with getting somewhere near that! I then did two tests where I copied to a drive outside of the array, and got around 175Mb/min. Next I did a copy to the unRAID array using read/modify/write and got 63Mb/min. Finally I did a copy to the unRAID array using reconstruct write and got 146Mb/min. So, I am happy that reconstruct write improves things quite significantly, so that's good. It's still some way off what should be possible though! Any insights or ideas? Cheers, SudoPacman Link to comment
SudoPacman Posted March 4, 2019 Author Share Posted March 4, 2019 Thought would try reading it back from the unRAID array, and that's certainly fine! Faster than the Drobo, assuming there is no caching going on.... Reading back, from unRAID to Desktop PCIe (2.0) SSD: ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 1 1 0 0 0 0 Files : 1 1 0 0 0 0 Bytes : 197.35 m 197.35 m 0 0 0 0 Times : 0:00:03 0:00:03 0:00:00 0:00:00 Speed : 56589191 Bytes/sec. Speed : 3238.059 MegaBytes/min. Ended : 04 March 2019 23:47:45 Link to comment
trurl Posted March 6, 2019 Share Posted March 6, 2019 On 3/4/2019 at 9:34 AM, SudoPacman said: The domain name was listed here: On 3/4/2019 at 9:50 AM, Niklas said: If you change something regarding your shares, this will be added to the syslog: I have started a discussion with the developers regarding further anonymizing diagnostics and directed them to this thread. Thanks Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.