August 25, 20196 yr Hello everyone, I am a new Unraid User, for now a Trial user. But I'm thinking to move to Plus version because right now I'm migrating my current storage system to Unraid. I did some tests and seems unraid cover all my needs. I'm in the middle of a migration from my current system to Unraid and I find out some issues regarding write speed drops. To be able to do this migration I'm moving the data from my current system to unraid formated disks in XFS, and No parity disks for the moment. The copy using any utility (rsync, rclone) drops from 100Mb/s to 10Mb/s When it takes about an hour and no longer recovers until I stop copying for a while and run the copy again. I have tried to copy directly to disk(/mnt/disk3), to share(/mnt/user) Also mounting the XFS disks manually(with the array stopped) All the cases I got the same results. If I solve the problem I will buy the plus license. since the system gives me everything I need Docker, VM, storage control, IPMI, etc ... This is my hardware specs: Supermicro X9DRi-LN4 + 2 Intel Xeon CPU E5-2650 2.00Ghz LSI Corp SAS2x36 LSI Corp SAS2x28 2 Adaptec - Adaptec RAID 71605 3 10TB HGST discs 18 8TB Seagate Archive Disks. Kind Regards, Jvdivx.
August 26, 20196 yr How are you copying? What is the source? Post the diagnostics. Ideally when this happens.
August 26, 20196 yr Community Expert 7 hours ago, jvdivx said: I have tried to copy directly to disk(/mnt/disk3), to share(/mnt/user) Please be aware of this problem: https://forums.unraid.net/topic/32836-user-share-copy-bug/#comment-316512 It is not a good practive to ever copy from a disk share to a user share or vice versa. The diagnostics file are generated by Tools >>> Diagnostics The file will be downloaded to the download directory/folder of your browser. Upload it with your next post. Edited August 26, 20196 yr by Frank1940
August 26, 20196 yr Author 6 hours ago, Frank1940 said: Please be aware of this problem: https://forums.unraid.net/topic/32836-user-share-copy-bug/#comment-316512 It is not a good practive to ever copy from a disk share to a user share or vice versa. The diagnostics file are generated by Tools >>> Diagnostics The file will be downloaded to the download directory/folder of your browser. Upload it with your next post. It was just a test to rule out problems with the array.
August 26, 20196 yr Author 11 hours ago, Squid said: How are you copying? What is the source? Post the diagnostics. Ideally when this happens. The disks are on the server connected to the same controller, source disk not assigned to the array, destination disk in the array. Source disk format btrfs (mixed) and target disk xfs format Example rclone working properly (For 1 hour and quarter) Transferred: 446.711G / 6.819 TBytes, 6%, 100.116 MBytes/s, ETA 18h34m5s Errors: 0 Checks: 386 / 386, 100% Transferred: 739 / 5331, 14% Elapsed time: 1h16m9s Transferring: * FolderX/***********: 98% /751.240M, 38.018M/s, 0s * FolderX/***********: 63% /751.004M, 30.781M/s, 8s * FolderX/***********: 88% /776.334M, 37.486M/s, 2s * FolderX/***********: 74% /748.467M, 26.427M/s, 7s The moment when you lower the write to disk to minimos and no longer recover. Transferred: 447.869G / 6.819 TBytes, 6%, 98.352 MBytes/s, ETA 18h53m51s Errors: 0 Checks: 386 / 386, 100% Transferred: 742 / 5331, 14% Elapsed time: 1h17m43s Transferring: * FolderX/***********: 20% /559.182M, 2.834M/s, 2m36s * FolderX/***********: 12% /559.518M, 2.286M/s, 3m34s * FolderX/***********: 95% /559.471M, 1.920M/s, 13s * FolderX/***********: 11% /558.863M, 2.042M/s, 4m3s jvdivx-unraid-diagnostics-20190826-1236.zip Edited August 26, 20196 yr by jvdivx
August 27, 20196 yr Community Expert Look at this Bug report and see if this might be involved in your problem:
August 29, 20196 yr Author On 8/27/2019 at 2:50 PM, Frank1940 said: Look at this Bug report and see if this might be involved in your problem: It's not my case. Only the 4 files are being copied to a disk and if I use rsync the same thing happens even if I write one file at a time. The funny thing is that it happens when it takes more than 1 hour and sometimes at 30 minutes. I have run out of 15 days license. If I turn off the array, can I continue trying some more time?
August 29, 20196 yr Community Expert 22 minutes ago, jvdivx said: I have run out of 15 days license. If I turn off the array, can I continue trying some more time? Trial is 30 days, and you can extend twice for 15 days more, so 60 days total.
August 29, 20196 yr Author 8 minutes ago, johnnie.black said: Trial is 30 days, and you can extend twice for 15 days more, so 60 days total. Thank you.
August 29, 20196 yr Community Expert On 8/25/2019 at 7:39 PM, jvdivx said: 2 Adaptec - Adaptec RAID 71605 I remember another using having performance issues with this controller, can you try a different controller, even if with just a couple of disks.
August 30, 20196 yr Author 5 hours ago, johnnie.black said: I remember another using having performance issues with this controller, can you try a different controller, even if with just a couple of disks. Unfortunately, I only have those controllers.
August 30, 20196 yr Author When it goes wrong It is seen that rclone processes disappear in the iotop. Although the rclone is still running. If I stop the rclone the situation is even more strange since no process is supposed to be writing on the disk but the iotop is triggered and is usually like this for 5 minutes. Edited August 30, 20196 yr by jvdivx
August 31, 20196 yr Author I have tried with version 6.6.7 and the exact same thing happens. Does anyone think of something? I am very interested in the product but if I do not solve it it would be a serious problem for my decision to make the purchase.
August 31, 20196 yr As you have SMR disk, could you try copying to a non-SMR disk. Edited August 31, 20196 yr by Benson
August 31, 20196 yr Author 3 hours ago, Benson said: As you have SMR disk, could you try copying to a non-SMR disk. The non-SMR discs, I already filled them and I had no problems. Even so, the friend who recommended me this system has the same disks as me and does not have these problems. If it were for the SMR I don't working well for more than 1 hour. What happens is that I have 18 SMR disks and in no system I had these problems. Very thanks.
August 31, 20196 yr Does mean, you identify problem relate SMR disk ? If any disk ( not all ) have slow write problem, then it will cause overall slow. Or you may further verify by plugin "diskspeed" to check any disk cause problem. Edited August 31, 20196 yr by Benson
August 31, 20196 yr Community Expert @jvdivx, many of us use use a SSD cache disk to handle the transfer of data to the array. There is a built-in mover which can be setup to move the contents of the cache drive to the array at a time when the server is idle. If parity protection of data is required even on the cache, you can use two or more SSD's in a cache pool to provide it. The maximum transfer speeds are much higher using SDD's because of the reduction due to head movement latency and rotational latency.
August 31, 20196 yr Author 2 hours ago, Benson said: Does mean, you identify problem relate SMR disk ? If any disk ( not all ) have slow write problem, then it will cause overall slow. Or you may further verify by plugin "diskspeed" to check any disk cause problem. Result of an SMR disk: Result of an non-SMR disk:
August 31, 20196 yr Author 2 hours ago, Frank1940 said: @jvdivx, many of us use use a SSD cache disk to handle the transfer of data to the array. There is a built-in mover which can be setup to move the contents of the cache drive to the array at a time when the server is idle. If parity protection of data is required even on the cache, you can use two or more SSD's in a cache pool to provide it. The maximum transfer speeds are much higher using SDD's because of the reduction due to head movement latency and rotational latency. I think that solution is not going to help me, I am doing a migration of more than 100Tb
September 1, 20196 yr 3 hours ago, jvdivx said: Result of everything the array has at the moment: Speed excellant, but plugin seems haven't write test, so can't identify any write issue on SMR dsik. When files writing to array, it will allocate to a disk, so could you try allcoate the write to another disk and check again. Next, I notice md_write_method was "auto" , may be you can try set to reconstruct write to see have any different. Edited September 1, 20196 yr by Benson
September 1, 20196 yr Author 4 hours ago, Benson said: Speed excellant, but plugin seems haven't write test, so can't identify any write issue on SMR dsik. When files writing to array, it will allocate to a disk, so could you try allcoate the write to another disk and check again. Next, I notice md_write_method was "auto" , may be you can try set to reconstruct write to see have any different. I also changed the value of md_write_method but it didn't improve much because I don't use parity-disk. I only write to one disk at a time.
September 1, 20196 yr 4 hours ago, jvdivx said: because I don't use parity-disk. Understand, I missing that. If problem confirm due to slow write on SMR disk, I don't think you could fixed by Unraid. If your friend no such problem, it may be just because he haven't trigger it, but you write several press on disk so trigger it. Edited September 1, 20196 yr by Benson
September 1, 20196 yr Author I have found a workaround while I find the definitive solution. Using the CA Users Scripts, I have scheduled a task to copy for 15 minutes and stop, and launch it again after 5 minutes. So I always get the top transfer I will adjust it if I see that I can extend the times more I have transferred 300Gb in 1 hour. Edited September 1, 20196 yr by jvdivx
Archived
This topic is now archived and is closed to further replies.