thevorticon Posted June 11, 2015 Share Posted June 11, 2015 Something is happening where if I'm copying a file via ssh, the disk speed will go to a crawl for 5-10 minutes, then speed up again like nothing happened. It also brings down the disk speed of my VMs/docker somehow. If I CTRL-C out of the rsync command I ran, it returns back to normal in a few seconds. I'm literally just bulk copying files around my disk array, and its usually fine, but is there some way I can find out what's behind this AWESOME slowdown that happens? Quote Link to comment
Squid Posted June 11, 2015 Share Posted June 11, 2015 post your syslog or if you're on rc5 your diagnostics report Quote Link to comment
thevorticon Posted June 11, 2015 Author Share Posted June 11, 2015 Recently upgraded to rc5, though this issue was present before upgrade. Report attached! edit: Just to clarify, the speed goes from 100+mb/s, to 20kb/s. Which in comparison is less than a snails pace Would love any help to diagnose what could be happening. tower-diagnostics-20150610-2223.zip Quote Link to comment
trurl Posted June 11, 2015 Share Posted June 11, 2015 Diagnostics missing smart for any of your hdds. Syslog has a lot of errors from your plugins, don't know if that's normal or not. Also says your cache drive is full, so that may be the problem. Quote Link to comment
thevorticon Posted June 11, 2015 Author Share Posted June 11, 2015 Diagnostics missing smart for any of your hdds. HDDs are connected through RAID card, probably why? Also says your cache drive is full, so that may be the problem. I opened another thread for that, it looks like my second drive in the pool either isn't getting used or is showing wrong size info. Quote Link to comment
trurl Posted June 11, 2015 Share Posted June 11, 2015 Looks like you have a cache pool with a 250GB and a 120GB. This means you only have 120GB usable. btrfs disk usage calculator Quote Link to comment
thevorticon Posted June 12, 2015 Author Share Posted June 12, 2015 Disk usage on cache lists about 33% full. Semi-doubt its because disk space but interesting theory. Quote Link to comment
trurl Posted June 12, 2015 Share Posted June 12, 2015 Disk usage on cache lists about 33% full. Semi-doubt its because disk space but interesting theory. How much space does it say is used, not in percentage but in GB? Jun 10 20:16:01 Tower shfs/user: cache disk full Quote Link to comment
thevorticon Posted June 12, 2015 Author Share Posted June 12, 2015 df -h says /dev/sdb1 173G 16G 95G 15% /mnt/cache Quote Link to comment
trurl Posted June 12, 2015 Share Posted June 12, 2015 Must be something wrong with your cache pool, corruption or something. 173G is definitely the wrong total. Should only be 120 on one disk and 120 mirrored on the other disk, so only 120 total. Quote Link to comment
thevorticon Posted June 12, 2015 Author Share Posted June 12, 2015 Is there an easy way just to reformat the pool? Quote Link to comment
trurl Posted June 12, 2015 Share Posted June 12, 2015 I think there were some similar reports a few months ago when people filled up their docker img. In Settings - Docker, what do you have under Docker volume info? Quote Link to comment
thevorticon Posted June 12, 2015 Author Share Posted June 12, 2015 I think there were some similar reports a few months ago when people filled up their docker img. In Settings - Docker, what do you have under Docker volume info? Label: none uuid: 6ab55325-6270-4669-a887-3da715f2bfc1 Total devices 1 FS bytes used 295.03MiB devid 1 size 10.00GiB used 3.04GiB path /dev/loop0 btrfs-progs v4.0.1 seems fine Quote Link to comment
trurl Posted June 12, 2015 Share Posted June 12, 2015 From Main, click on the cache drive (not cache2). On the Cache Settings page, turn on Help. Scroll down and read the help under Scrub and click the button. Quote Link to comment
thevorticon Posted June 12, 2015 Author Share Posted June 12, 2015 Scrub came back with 0 errors scrub status for df489f54-c8eb-48e1-b5a9-54440fc6fa92 scrub started at Fri Jun 12 18:10:38 2015 and finished after 37 seconds total bytes scrubbed: 31.73GiB with 0 errors Quote Link to comment
trurl Posted June 13, 2015 Share Posted June 13, 2015 Maybe try balance. Shooting in the dark here. Quote Link to comment
thevorticon Posted June 13, 2015 Author Share Posted June 13, 2015 Tried it, same result. I'm gonna switch it to just my 256gb SSD instead of the 2 and see what happens Quote Link to comment
thevorticon Posted June 14, 2015 Author Share Posted June 14, 2015 Minor update: Appears to happen with only 1 cache drive (reformatted) as well. I'm noticing that the next drive in sequence never gets spun up even though the current drive is half full (share is set to high water). Maybe this is where it gets hung up? Though I guess if its just writing to itself (the cache drive) until the mover hits, this doesn't seem likely. Attempting to have all drives spinning to see if I can replicate. Quote Link to comment
thevorticon Posted June 15, 2015 Author Share Posted June 15, 2015 Update 2: I just got a "heat warning" for my cache SSD, says its running at 45C. I guess this could be it too? This SSD used to run 5-6 VMs at one point with no problem so I sort of doubt it unless docker/plugins are doing something weird. Edit: Nvm it just went back to 27C Quote Link to comment
thevorticon Posted June 15, 2015 Author Share Posted June 15, 2015 Bumping It appears to occur when I write to the cache/SSD, I was copying some files off an external drive to my array and the slowdown/crawl occured. BUT, READING from the drive or writing at a low (under 1mb/s) speed seems to not cause it to occur. Wondering I should format from btrfs to xfs on the cache and see if I get similar results? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.