landS Posted February 2, 2015 Share Posted February 2, 2015 Perhaps my google foo is weak... What happens when my xfs data drives fill up, but I still have 1 TB of raid-1 btrfs cache drive (6b12)? Will unraid allow me to transfer to the cache drive, but since the data drives are full (as per min free space set on share), prevent the mover from moving? (Note: I want to do this as I wait to finish pre-clearing a 4TB hgst data 3 times) Link to comment
garycase Posted February 2, 2015 Share Posted February 2, 2015 I don't know the answer to this ... but I'd simply wait for the new drive to be added and then transfer your data. If you're "anxious" to get the data on the server, just create a cache-only share and copy the data to it -- you can later move the data to an array disk and delete the extra share. But I'd just be patient, and wait for the new 4TB drive to be added before copying that data to the server. Link to comment
afoard Posted February 2, 2015 Share Posted February 2, 2015 I believe it depends on your share settings. If you have a share marked to use the cache drive, then any file will still be transferred to your cache drive and will live there until you add another data drive. If you do not use a cache drive for a certain share, you would most likely start getting a bunch of errors when you or any program attempts to move files directly to the array. Link to comment
SSD Posted February 2, 2015 Share Posted February 2, 2015 I don't use the cache drive to cache my writes. But in the scenario you portray, if the share is set to be cached, I believe that the mover will still try to move the file to one of the disks in the share even if less than min-free space remains. Presumably it will fail and the file will remain on the cache drive until the next night when it will try again. Link to comment
landS Posted February 2, 2015 Author Share Posted February 2, 2015 Thanks crew - I am anxious as setting up this backup server timeline was accelerated by at least 2 months. I had planned on the backup server taking over the duty of BACKUP data and the main server taking over the duty of PRIMARY data. A bad brownout killed my main windows rig, the cyberpower UPS it was on, and none of the 8 drives (all 3 TB HGST) are readable now. My primary data lived here on raid-1 disks --- all backed up to my main unraid machine. This machine was to be decommissioned as a data storage device, but only after getting the backup unraid server up and running 100% This means that my main unraid machine now contains the only copy of my data. On this backup unraid machine all of my shares are setup to use the cache disk. My main server has NO cache disks. Link to comment
garycase Posted February 2, 2015 Share Posted February 2, 2015 I'd assume the same behavior ... but I also see little reason to bother doing the copies until the new drive is simply added to the array Link to comment
landS Posted February 2, 2015 Author Share Posted February 2, 2015 will do - moving over most important data to remaining array capacity. Will stop when the level is reached and add 4 TB when able (15% into first post-read) Link to comment
garycase Posted February 2, 2015 Share Posted February 2, 2015 ... A bad brownout killed my main windows rig, the cyberpower UPS it was on, and none of the 8 drives (all 3 TB HGST) are readable now. My primary data lived here on raid-1 disks --- all backed up to my main unraid machine. You lost EIGHT disks all at once?? ... from a brownout? Was this a cheap UPS without AVR?? In any event, this is the kind of thing that clearly shows why you need backups !! (Glad you had them) ... This means that my main unraid machine now contains the only copy of my data. Risky ... as I'm sure you know. I'd be sure parity's good (check it if you haven't recently) ... and would simply shut down the server if you get any drive failures before the backup server is ready for the copies. Link to comment
garycase Posted February 2, 2015 Share Posted February 2, 2015 This incident also shows why it's a good idea to keep a fully tested spare handy. Worst case is you end up with a drive that's a bit smaller than you'd like (if it isn't needed until higher capacity drives are common) ... but it can always be used as a backup drive; a storage drive for a workstation (where you don't need the highest capacity drives), etc. Link to comment
trurl Posted February 2, 2015 Share Posted February 2, 2015 The cache drive is not protected by parity so in one sense there is little reason to use it to cache data if that data can't be written to the array due to lack of capacity in the array. However, if you don't have a backup of the data, then writing it to the cache could be thought of as making a backup. You can change the mover schedule in the webGUI so it doesn't try to do the move until you are ready. Link to comment
landS Posted February 2, 2015 Author Share Posted February 2, 2015 My servers are both on an APC Smart UPS 1000. I moved the Windows machine to a cheaper cyberpower UPS (and purchased fresh batteries for the backup servers UPS) when I began getting the backup server up and running in-place a few weeks before the really bad power issues. It is the LAST time I mess with a cheap UPS and should have let the APC pull double duty for the windows/backup machine until I was done. The PSU is a Seasonic gold model - it should have been fine. I have tried another PSU in the machine, an external dock, and even an esata hookup to each of the disks - none of the disks are readable. This brownout also killed my: septic ejector, some LED light bulbs, the living room TV (on a belkin power conditioner), and our washing machine. This IS risky and I am not comfortable with this. The last parity check was my Jan 1st (all good) - EVERY parity check has been error free for years, the power went out on Feb 1st check, but manually relaunched and it should be done within 23 hours. I WILL shut it down if it has ANY errors, but 8% in and all looks good. My MAIN server has a fully precleared ready to go 4 TB spare - but I do NOT want to yank that for the new backup server. This backup machine has a brand new 4 tb hgst for parity, and currently is using a mix of older precleared drives for data. It would have gone into production with all new 4 TB hgst data drives, but for this opera. Link to comment
landS Posted February 2, 2015 Author Share Posted February 2, 2015 @trurl - the cache is a set of WD Red (lightly used, triple precleared) 1 TB disks in a BTRFS Raid-1 cache pool. While not protected by parity, the data is at least mirrored. Link to comment
garycase Posted February 2, 2015 Share Posted February 2, 2015 ... My MAIN server has a fully precleared ready to go 4 TB spare - but I do NOT want to yank that for the new backup server. Yes, I saw that in your other thread ... and would have suggested you use it except for your comment about the accessibility of it. Link to comment
garycase Posted February 2, 2015 Share Posted February 2, 2015 The cache drive is not protected by parity so in one sense there is little reason to use it to cache data if that data can't be written to the array due to lack of capacity in the array. Actually his cache drive IS fault-tolerant, as he's using a btrfs cache pool in RAID-1 Link to comment
garycase Posted February 2, 2015 Share Posted February 2, 2015 ... While not protected by parity, the data is at least mirrored. Fault tolerance is fault tolerance. A mirror provides single failure fault tolerance, just like the parity drive does for the array. From a fault tolerance perspective there's really no difference. Link to comment
landS Posted February 2, 2015 Author Share Posted February 2, 2015 ok - leaning towards an attempt to disable mover scheduler for now, and re-enable mover after 3 preclears of 4 TB HGST. ... back to searching Link to comment
BobPhoenix Posted February 2, 2015 Share Posted February 2, 2015 ok - leaning towards an attempt to disable mover scheduler for now, and re-enable mover after 3 preclears of 4 TB HGST. ... back to searching I took the brute force method since my searching the forums was lacking. I was able to find out where mover was located (sorry at work now so would have to search for that again). So I just renamed the mover script which should effectively turn it off. Are there better ways - I'm sure there are but since I couldn't seem to search for it I just disabled it. My problem was I wanted the files on the share to show up under USER shares but I didn't want mover running until I had everything generated (mkv's made from the BluRay disks). Link to comment
trurl Posted February 2, 2015 Share Posted February 2, 2015 The cache drive is not protected by parity so in one sense there is little reason to use it to cache data if that data can't be written to the array due to lack of capacity in the array. Actually his cache drive IS fault-tolerant, as he's using a btrfs cache pool in RAID-1 Sort of makes the point of my 2nd paragraph even more relevant then. I don't have a v5 to look at anymore, but on v6b12 the mover schedule is at Settings - Global Share Settings - Mover Settings. Doesn't look like you can actually turn it off, but you can set it to weekly and choose a day of the week or monthly and choose a day of the month and so make it far enough in the future that you can finish the preclears. Link to comment
Squid Posted February 2, 2015 Share Posted February 2, 2015 ok - leaning towards an attempt to disable mover scheduler for now, and re-enable mover after 3 preclears of 4 TB HGST. ... back to searching I took the brute force method since my searching the forums was lacking. I was able to find out where mover was located (sorry at work now so would have to search for that again). So I just renamed the mover script which should effectively turn it off. Are there better ways - I'm sure there are but since I couldn't seem to search for it I just disabled it. My problem was I wanted the files on the share to show up under USER shares but I didn't want mover running until I had everything generated (mkv's made from the BluRay disks). While its not exactly disabled, why not just change the schedule to monthly and set the day of the month to the 31st? It shouldn't run during February Link to comment
landS Posted February 2, 2015 Author Share Posted February 2, 2015 @bob - this solution works fine for me! Good idea @trurl - this is 6b12 - I missed that! Still exploring 6 and what you wrote did not register I will just set it for monthly (28 days out) - after all preclears are run I will reset it to nightly. THANK YOU ALL AGAIN! Link to comment
landS Posted February 2, 2015 Author Share Posted February 2, 2015 You can change the mover schedule in the webGUI --- found it settings/global/mover Link to comment
landS Posted February 3, 2015 Author Share Posted February 3, 2015 So.... EEK. Saved Mover schedule to monthly and 31st, checked change post apply (clicked on cache and back on mover under global) last night and all looks good. This AM it was back to the default. When no more disks are present Mover ignores minimum share space. I have my share set to 70 GB min and the disk currently sits at 22 GB free. This seems like somewhat scary beahviour, no? Putty into the system, logged in, typed killall rsync yet the mover kept chugging along. Attempted to stop array in GUI... it didn't Had to force power down - hopefully didn't screw up any files mid-move. Running a parity check given the forced shutdown - if any errors will run a 2nd parity check for sanity sake. edit: mover scheduler persists the 1st, doesn't persist the 31st. edit2: dang - 4 TB HD was 98% post read on 2nd preclear attempt... almost 80 hours of 2 nearly finished preclears... gosh dangit. Link to comment
SSD Posted February 3, 2015 Share Posted February 3, 2015 I don't use the cache drive to cache my writes. But in the scenario you portray, if the share is set to be cached, I believe that the mover will still try to move the file to one of the disks in the share even if less than min-free space remains. Presumably it will fail and the file will remain on the cache drive until the next night when it will try again. Did you read this? You should not have been surprised. Link to comment
landS Posted February 3, 2015 Author Share Posted February 3, 2015 Yes - but what surprised me is that Mover scheduler setting did not persist when I set it to the 31st 1 - set to monthly/31st/apply, click on gui CACHE under global and click on Mover - setting was the 31st 2 - check this AM and mover under global back to daily 3 - set to the 1st, powercycle the machine, and under Global it still shows the 1st. After 4 TB disk is in the array I will delete the directories currently split between the Cache and the Data2 drive and re-robocopy. All should be good. A note to future readers: I am forcing this machine to do some non-standard use while I preclear a larger datadrive and recover from a bad drive (that actually passed 3 preclears). In normal use (at home, and multiple at work builds) unraid is rock solid! Link to comment
BobPhoenix Posted February 3, 2015 Share Posted February 3, 2015 ok - leaning towards an attempt to disable mover scheduler for now, and re-enable mover after 3 preclears of 4 TB HGST. ... back to searching I took the brute force method since my searching the forums was lacking. I was able to find out where mover was located (sorry at work now so would have to search for that again). So I just renamed the mover script which should effectively turn it off. Are there better ways - I'm sure there are but since I couldn't seem to search for it I just disabled it. My problem was I wanted the files on the share to show up under USER shares but I didn't want mover running until I had everything generated (mkv's made from the BluRay disks). While its not exactly disabled, why not just change the schedule to monthly and set the day of the month to the 31st? It shouldn't run during February I would think that would work for any month less than 31 days. If I knew what file and/or line to edit manually I would change it to 32 and turn it off for all months. Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.