pras1011 Posted May 25, 2015 Share Posted May 25, 2015 I am trying to transfer lots of data from old disks to new disks within the server. The write speed is so slow. Therefore I want to disable parity to improve write speed. What's the EXACT step by step procedure in V6 to do this? Quote Link to comment
garycase Posted May 25, 2015 Share Posted May 25, 2015 The best way is to do a New Config and simply don't assign a parity drive ... i.e. assign all the data drives exactly as they are now -- and the cache drive if you have one -- but do NOT assign a parity drive. Then, when the writes are all done, you simply Stop the array; assign the parity drive; and then Start the array again -- it will then do a parity sync. As I'm sure you're aware, doing it this way means your array won't be fault tolerant while you're writing the data to it => so be sure you have a complete set of backups of your data (should always have those anyway, but clearly it's even more important when you don't have a parity drive). Quote Link to comment
RobJ Posted May 25, 2015 Share Posted May 25, 2015 I think Gary hadn't had his coffee yet! Easy way is to stop the array and unassign the parity drive. Period. Once you start the array, and make a single write to the array, the parity drive will no longer be valid. Quote Link to comment
garycase Posted May 25, 2015 Share Posted May 25, 2015 Yes, that will work too => as long as you don't mind the "Missing" warning on the Web GUI. I like it "clean: But since you're not actually likely to even look at the GUI, it's certainly a quicker way :) (Rob's right -- I wasn't completely awake when I posted that) Quote Link to comment
pras1011 Posted May 26, 2015 Author Share Posted May 26, 2015 Thanks all!! Speed is not that much faster! Quote Link to comment
garycase Posted May 26, 2015 Share Posted May 26, 2015 Thanks all!! Speed is not that much faster! Interesting => I wonder if there's a difference in how UnRAID treats an array with "missing parity" vs. writing to an array that simply hasn't had a parity drive assigned [e.g. a "New Config" with no parity assigned]. Might be worth an experiment if you still have a lot of data to copy. Quote Link to comment
pras1011 Posted May 26, 2015 Author Share Posted May 26, 2015 Due to all this frustration, no more experimenting now! I have gone from 23tb data to move down to 6tb! Removing parity is pretty extreme for me. Quote Link to comment
garycase Posted May 26, 2015 Share Posted May 26, 2015 Agree => 6TB is really not all that much data to move ... just takes a bit of patience. [Personally, I'd have just moved it WITH parity enabled -- doesn't take any more of "my time" -- just the computer's] Quote Link to comment
pras1011 Posted May 26, 2015 Author Share Posted May 26, 2015 The reason why I disabled parity (I was going to have to do it sooner anyway to remove the spare drives) was that whilst moving data the speed would suddenly drop to zero and the parity was continuously active. And this would timeout the transfer. Quote Link to comment
garycase Posted May 26, 2015 Share Posted May 26, 2015 Ahh ... I think you have "hit the wall" on the shingled drive performance with your copies !! The good news is once the data's all copied, it's unlikely you'll encounter that issue again with normal array usage. Good to know actually => when I build my next BIG array I'll remember this and just copy my entire media server collection to the new array BEFORE assigning a parity drive. Quote Link to comment
pras1011 Posted May 26, 2015 Author Share Posted May 26, 2015 Agreed. But on some massive transfers there was absolutely no problem. All strange. Quote Link to comment
garycase Posted May 26, 2015 Share Posted May 26, 2015 One notable difference between your transfers and the testing Daniel did is (I think) that you have several more data drives. This likely caused the writes to the parity drive to be "scattered" a lot more, and thus may very well have caused the persistent cache to fill up quicker ... thus "hitting the wall". This is EXACTLY the scenario that made me (and others) very skeptical about using a shingled drive for parity. I still think Daniel's testing shows that for MOST typical UnRAID array usage patterns, it's not likely to be a problem => indeed as long as you never copy more than 25GB at once it certainly won't be, since the persistent cache will have time to empty itself during the interim idle periods ... but clearly for a very active array with a lot of writes, the concerns are very valid. The idea of using a pair of 4TB drives in a RAID-0 array for your parity drive in arrays with the 8TB SMR drives is perhaps not a bad one if your usage pattern is one that may have this problem. Quote Link to comment
pras1011 Posted May 26, 2015 Author Share Posted May 26, 2015 Like you say once it's all transferred then I will never need to do this again. raiding parity makes me nervous. I had about 13 data hdd and a parity drive. Unfortunately it timeout again. Quote Link to comment
garycase Posted May 26, 2015 Share Posted May 26, 2015 ... Unfortunately it timeout again. Your experience is a bit scary ... not nearly as positive as Daniel's. What are the characteristics of your data? i.e. all large media files; a lot of smaller files; all going to the same place, or are they spreading around a lot due to share settings/split values ?? Quote Link to comment
pras1011 Posted May 26, 2015 Author Share Posted May 26, 2015 I am doing a direct disk to disk transfer of large bluray files. I am transferring from hd share disk to disk. I am now doing smaller transfers and then swapping to another target data disk to give the previous disk a break to clear the persistent cache. The timeout disk only has 500gb left of the 8tb and maybe that's the problem. Quote Link to comment
garycase Posted May 26, 2015 Share Posted May 26, 2015 The timeout disk only has 500gb left of the 8tb and maybe that's the problem. Hmm ... I wonder if this is the same issue with XFS that Reiser has when a disk is very full => if so, this may be a file system issue and not an SMR problem. Really hard to say for sure, but I'd think that a large BluRay file would result in full band writes that wouldn't use much (if any) of the persistent cache. Quote Link to comment
pras1011 Posted May 26, 2015 Author Share Posted May 26, 2015 I was having this problem with my old hitachi 3tb drive. It would timeout straight away and then work in the second try. I would hate to see what would happen on a 20tb smr drive!! And preclearing will be a pain! Quote Link to comment
garycase Posted May 26, 2015 Share Posted May 26, 2015 If it was happening with a non-SMR drive (the 3TB Hitachi) it may very well be a file system issue and have nothing to do with the shingled nature of your drives. Quote Link to comment
pras1011 Posted May 26, 2015 Author Share Posted May 26, 2015 I tried it on windows 10 with a dell e5420. But it didn't work. Transfer seems a little faster. Quote Link to comment
mondama13 Posted February 4, 2021 Share Posted February 4, 2021 I had 6 drives in my array without a parity drive for over 2 months. The speed difference is over 2x without parity enabled. When I used Krusader to copy large movie files (>20gb) from one drive to another, I used to average about 150 MB/s, but now with a single parity drive my transfer speed is about 55 MB/s. I wonder if its possible to turn off Parity when you want to and then enable it later to run in the background and sync everything. Anyone know if the speed is any better with 2 parity drives? Quote Link to comment
Energen Posted February 4, 2021 Share Posted February 4, 2021 7 minutes ago, mondama13 said: I had 6 drives in my array without a parity drive for over 2 months. The speed difference is over 2x without parity enabled. When I used Krusader to copy large movie files (>20gb) from one drive to another, I used to average about 150 MB/s, but now with a single parity drive my transfer speed is about 55 MB/s. I wonder if its possible to turn off Parity when you want to and then enable it later to run in the background and sync everything. Anyone know if the speed is any better with 2 parity drives? I don't believe there is any way to disable parity except to unassigned the parity drives. I agree the speed issue is annoying but the trade off is you disable parity to move the data and then have to create parity each time you do this. That might not be as annoying since you can do parity in the background but my parity checks/creations take around 8-10 hours, and I'm uneasy about writing new data to the array during that time, but that's just me. So is a slower transfer better or a long parity creation time? That's your choice. 2 parity drives does not effect the speed in either way. Quote Link to comment
JonathanM Posted February 4, 2021 Share Posted February 4, 2021 7 minutes ago, mondama13 said: I wonder if its possible to turn off Parity when you want to and then enable it later to run in the background and sync everything. Not really. I guess you could do a new config and remove the parity assignments, then do another new config, add parity back, and rebuild it. That would take much longer to complete and you would be unprotected from drive failure until parity was done building. 10 minutes ago, mondama13 said: Anyone know if the speed is any better with 2 parity drives? Definitely no. In some instances it's slower because parity 2 slot requires much more maths, and some CPU's don't have the horsepower. Why are you doing so much disk to disk moving? Normally that's not really a thing, unless you are rethinking how you want things permanently arranged. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.