tillkrueger Posted June 20, 2015 Share Posted June 20, 2015 as i briefly chronicled in this thread (http://lime-technology.com/forum/index.php?topic=40788.0) i was under the impression that there was some sort of issue with how my new cache drive behaved when i saw backup speed drop from just under 30MB/s to under 4MB/s in under a day...so i ended up canceling the backup from my USB2 drive (connected to my iMac, which is connected to unRAID via 1Gb ethernet) and started the same operation directly to my array after disabling the cache drive again. when i saw it copy at 27MB/s yesterday, i thought that things were going to be ok, but i am now looking at 3,82MB/s again, over 22hrs into trying to copying a 305GB sparsebundle backup...needless to say, since just for this backup i need to copy over 2TB of data, this is totally unacceptable. i am so frustrated right now...unRAID has been a loyal and dependable friend these past 10+ years, and my server safely (so far) holds over 21TB of my production data from the past 20 years...but it seems that these past 6 months i've stumbled from one issue to the next, which have cost me a ridiculous amount of hours of time to trouble-shoot, test, fix and keep trying...whatever i try, i just don't seem to be able to get back to the "set it and forget it" ease that made me stick with unRAID for so long. chances are, it's user error or maybe some weird configuration or even hardware problem, but where do i even start? i ordered an external OWC Thunderbolt 2 RAID encore with Softraid software to begin migrating some of my more important production data to a faster array that i can more easily take with me than this 15-Bay behemoth that my unRAID server has become, but even with a fast external and portable RAID5 array, i still need and want unRAID to be my safe for all i have done and will do. where do i start with figuring out why i am now copying at under 4MB/s...it's really maddening to keep banging my head against the wall with this...help!! Link to comment
Squid Posted June 20, 2015 Share Posted June 20, 2015 Post the diagnostics from the system (Tools/Diagnostics) so we can see if anything is amiss there (ideally grab the diagnostics while transfer speed is abnormally slow) Link to comment
tillkrueger Posted June 20, 2015 Author Share Posted June 20, 2015 to rule out the possibility that ChronoSync might have anything to do with it, i did a direct copy in Finder, from the folder on the attached USB2 drive to one of the disk shares (also trying to rule out that it's a user share issue), and it took about 30 seconds until the copy progress window showed 8MB being copied, and it said "13 days" as an estimate for the same 306GB sparse bundle file...i waited another 10 minutes, and it now copied 192MB, dropping the estimate to 11 days...not good at all. as per your request, Squid, attached please find the diagnostics. i sure hope this situation can be fixed. of course, copying from an iMac attached USB2 drive to the unRAID array via the OS X Finder is sub-optimal, but since i don't have physical access to the system over the weekend, i can't plug the external drive into one of the unRAID's USB ports and try to use the Attached Devices plugin and Midnight Commander (as i am planning to) to rule out the Mac being an issue...what i probably should do, though, is try to reboot the iMac, which i am always a bit nervous to do for fear of it not "coming back" and me then losing remote access to both the iMac and the unRAID system...but since there isn't much i can do with this type of copy speed right now anyway, i might as well take the risk...here's to hoping... unraid-diagnostics-20150620-2041.zip Link to comment
Squid Posted June 20, 2015 Share Posted June 20, 2015 Were those diagnostics from during (or after) it was super slow or before? Link to comment
tillkrueger Posted June 20, 2015 Author Share Posted June 20, 2015 During what I just described in my last post. Link to comment
Squid Posted June 20, 2015 Share Posted June 20, 2015 The logs look good. No obvious problems anywhere. You cache drive looks good. Only thing I can think of is either your Sab plugin or a docker container stealing all of the CPU cycles and lowering your transfer speed. (Or try rebooting the Mac to (hopefully) eliminate any source problems) Link to comment
tillkrueger Posted June 20, 2015 Author Share Posted June 20, 2015 sabnzbd is running but not doing anything...unRAID's CPU utilization is currently at 3-8%. i'll do a reboot as soon as a Dropbox sync i started on the iMac is finished (about 30mins), then try the copy operation again...thanks for looking at the diags, Squid. Link to comment
garycase Posted June 20, 2015 Share Posted June 20, 2015 ... my server safely (so far) holds over 21TB of my production data from the past 20 years ... Just to be sure, you DO have this data backed up somewhere else .. right?? Link to comment
garycase Posted June 20, 2015 Share Posted June 20, 2015 A couple questions r.e. the large files you're trying to copy .... (a) Do these files already exist, or are they being created on the fly by ChronoSync? i.e. does it give a size estimate of what the backup will require; or has it already compressed all of your backups into a single large file, and that's what you're trying to copy? (b) Are you copying this to a drive that has plenty of space for the file? © Have you been using this same process successfully in the past and this issue has just now cropped up ... or is this a new way of doing things? (d) If you've been using the same process, did it work fine with UnRAID v5? Just trying to isolate what the potential problem areas are. From your previous thread, it seems the big "New" item is that you upgraded to v6. If it worked fine in v5, then unless you've done something that makes reverting impossible (i.e. changed your file system) I'd try going back to v5 and seeing if the issues goes away. ... But my key concern -- given that you've got 20 years of important data on the system -- is that you've got good backups before you "mess" with anything. Link to comment
tillkrueger Posted June 20, 2015 Author Share Posted June 20, 2015 you mean those 21TB of data? i do have the most important 10-15TB of it backed up elsewhere, but this is the only complete and current version of all that data...i am well aware of this being sub-optimal, but once i get the external Thunderbolt 2 RAID5 populated with disks later this summer, i will copy the newest version of those 10-15TB to it...hopefully at better speeds than 4MB/s if i can't get this slow-speed situation figured out soon, i'll likely just shut down the unRAID until i get the external RAID5 set up, and only start unRAID up if i can't find the data i need elsewhere...but frankly, i can't imagine that this situation with my unRAID will end in massive data loss...yes, crazier things have happened, but i'm inclined to believe that this is a solvable speed issue and nothing else. but thanks for being cautious, graces...you've always been a valuable voice of wisdom and knowledge here and i trust your instincts implicitly. i saw you just posted a number of questions...i'll try to answer those now... a) these are a few sparsebundles, which are single-file system backups (like dmg files that can be written to), created by Carbon Copy Cloner b) the file in question is 306GB, going to a drive that has 700GB available...when i first tried this, though, i copied straight to the user-share, which has 12TB available, so i don't know which drive unRAID picked to copy to. c) i have often used ChronoSync without these issues to copy to unRAID...and since i just found a direct copy through Finder to be even slower, i think that i ruled out ChronoSync being the culprit. d) the process has worked fine in the past, although i am not sure whether i have copied sparsebundles of this size before...since sparsebundles are really entire system backups in a single file, they often contain hundreds of thousands of files, so maybe that has something to do with it...i shall try a more "straight forward" copy operation with a single file, like a movie, to see whether there is a difference...but i don't see why the difference would be that big, even if there is some indexing that is taking place while copying a single-file system backup. if nothing else works, i guess i will have to go back to v5 for testing, but i'm already struggling to keep up with all of my to-do items right now and am cringing at the thought of spending days with my unRAID system yet again...of all people you know what i've been through with this system since i brought it with me from California and replaced all of the drive cages, graces...you helped me with that trouble-shooting process quite a bit as well, if you remember. Link to comment
garycase Posted June 20, 2015 Share Posted June 20, 2015 ... since sparsebundles are really entire system backups in a single file, they often contain hundreds of thousands of files, so maybe that has something to do with it ... That's why I asked. If there are "hundreds of thousands" of individual files that are part of this copy, that easily explains why everything's slowing down so much => EACH file results in a directory update, which causes 4 more disk access and seeks to the directory area and back to where the file is being written. This is FAR slower than writing a single large file. If the file is simply a container that holds those hundreds of thousands of files (i.e. like a .Zip archive) then it should copy just fine ... but if it's actually storing that many individual files it's easy to understand why it's so slow. ... and yes, I remember well your travails after your move from California Link to comment
garycase Posted June 20, 2015 Share Posted June 20, 2015 By the way, the Thunderbolt 2 RAID-5 box is a VERY nice storage choice. Write speeds better than 10x what you get with UnRAID. I suspect you'll end up using that as your primary archive with the UnRAID box relegated as a backup. On the other hand, the comments I made in your other thread about USB2 not really being a bottleneck with UnRAID clearly don't apply with the Thunderbolt Might be time for a new Mac with USB v3 :) Link to comment
tillkrueger Posted June 20, 2015 Author Share Posted June 20, 2015 ok, well, that's encouraging...always better to have a name for the disease, than simply a non-descript ailment in that case i can probably live with keeping my sparsebundles on loose drives and put them in the shelf, rather than gunkning up my unRAID with them and dying in the process of watching them copy to it...problem solved...i'll do that instead...good way to use that big stack of 1 and 2TB drives that's been building in my tech-cabinet over the past few years...i very rarely ever have to go back to those system backups for any reason anyway, so it's really no loss. and yeah, i am *very*excited about that OWC case coming later this summer...for money-saving reasons i had to order it from a USA vendor for shipping to my ex-girlfriend's place in Berkeley...she'll bring it back with her when she comes back to Berlin in late July...an excruciatingly long time to wait for it, but it saves me almost $400 over buying it here in Germany with Euros (and the version i ordered actually includes the SoftRaid software)...i read about all the various solutions for many hours before settling on the OWC case with SoftRaid...can't wait! as for USB2 vs USB3, during a recent video-editing project that had a good budget and profit margin, i gave up on my iMac 5K i5, which kept crashing and giving me GPU problems (Apple seriously messed up by choosing to sell that machine with that 290 GPU), and sold the i5 to my father (who doesn't do any 3D or video editing) and sprung for the i7/295 version of it, which absolutely screams on every level...so, i am more than ready for that Thunderbolt 2 case Link to comment
tillkrueger Posted June 20, 2015 Author Share Posted June 20, 2015 i know this isn't the hardware forum, but since i haven't gotten a single response to my thread there, and since i know that you know a lot, garycase, would you be able to tell me what USB3 PCIe x1 controller i need to be looking for for my unRAID server, so that i can get data off it via direct attached USB3 storage and Midnight Commander? what chipset am i looking for on that USB3 card? the ones that i saw mentioned people using aren't available anywhere i searched here in Germany. reading off the array via Midnight Commander to a USB3 drive should be pretty quick, no? bc i'll need to copy a LOT of data (at least 10-15TB). Link to comment
garycase Posted June 20, 2015 Share Posted June 20, 2015 I really don't know for sure which chipsets UnRAID supports, as I've not added USB3 to my server. I do everything across the network from my PC. I did add a USB3 card to my PC using the HooToo HT-PC002 card, which works very well. It uses the VIA VL800 chipset, which I do NOT know if Linux supports. But there's a post in this forum that indicates the Mediasonic HP1-U34F works just fine, so that's what I'd look for. I have no idea what they cost over there (or even if they're available ... but they're only $16.99 at Newegg -- actually even cheaper this week with a $5 promo code). That card apparently uses the FRESCO LOGIC FL1100-1Q0 chipset. Link to comment
tillkrueger Posted June 20, 2015 Author Share Posted June 20, 2015 i found this card which appears to be built on the FRESCO LOGIC FL1100 chipset (not sure how important the "-1Q0" part is?: http://www.amazon.de/Stromanbindung-Inateck-KT4006-expresskarte-erforderlich/dp/B00JEVLEFQ/ref=sr_1_8?ie=UTF8&qid=1434837887&sr=8-8&keywords=FRESCO+LOGIC at that price i may just order and try it...nice thing with Amazon is the ease with which items can be sent back, even here in Germany. thanks, garycase! i'm off to bed... Link to comment
garycase Posted June 20, 2015 Share Posted June 20, 2015 Looks like a good card ... and it's certainly inexpensive enough. Plus, as you noted, Amazon's return policies are very good (good to know that's also true internationally). Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.