axeman

Thanks ... I ended up doing it in a VM. I have the accounts created.. honestly, I feel like a script kiddie on this one. I normally understand what I'm doing, but with this, I'm just copy/pasta. Slowly but surely, getting there.

hi - when creating the service accounts from the optional section... we can do that on another machine other than UnRaid, right? I'm thinking of spinning up a VM just to get the accounts created.

Thanks, I ended up going back to ESXi.

yeah - I agree we are certainly nothing in the grand scheme of things. Just worried about not having a fallback plan. Just like how Wink is not extorting it's users. I am going to get started, at least at first with a copy mode. I will have actual, real, script related questions when I start down the path. Once again, thank you for your time, and willingness to share.

Okay - I think I'm over complicating it. Will probably just keep it in copy mode, use the UnRaid shares and then decide what to do from there. Worst case if a drive locally fails, I can make a decision to either manually restore it, or just let the cloud hold that drives data, and shrink the array. Perhaps like you say, eventually i'll be all cloud and only have a small array for irreplaceable data. My fear is that Google being the only bastion of hope, puts a limit, or restriction of some sort (like Amazon did) and having to build an array up before the data gets deleted.

Yeah, I went through the Readme... and even glanced through the code and still not sure how it all works. Want to get a good understanding before jumping in. Thanks! That would be crazy if i can save some wear/tear on my whole array. So in my use case, I want this to be sort of a second (exact) copy of my array. This way, when I have a disk failure, I can just redownload all the data for that disk. I know that's really not the goal of this project, but I think having a backup of my array, in the "cloud" AND being able to stream off that is some crazy cool stuff. Yes... sort of, at least in my use, I'd still maintain my UnRaid local array. Either way, if/when I get this working, I'll be sending something your way. Great - You should post that on GIT ... Probably won't make you a millionaire, but hey, a little bit is better than nothing.

First big Thanks to @DZMM on posting this here. Could've easily just kept it to yourself. Sharing is caring! I've been reading bits and pieces of this topic, and admittedly have not gone through all 68 pages, so please forgive me if these have been covered. My media tool will be Emby. My primary purpose for this would be to use it as sort of a backup for my physical media in the house. Ideally though, when requesting a show or movie, priority would be given to the uploaded data instead of my local drive spinning up. So it's almost like my UnRaid array would the backup for the cloud data. I saw one of the replies mention setting the option to copy instead of move. I'm guessing that's answers a part of this requirement, but is there a way to make the uploaded data preferential? Is it just a matter of NOT using MergerFS and just using the rclone vfs mount? The trickier part is, I'd like to upload the data as Disk shares However, my Emby server is set to find media via the user shares (Videos, TV Series, etc). Is this too complicated of a scenario? Reason for this is, if I have a local disk failure, I'd want to re-download the data that's been pushed up to Gdrive. It's a lot easier to just grab everything from Disk 3 instead of trying to rebuild from directory structure. I juggle my data around a lot between disks. Will this end-up with multiple versions on GDrive, or will it just be recognized as a move? Finally @DZMMWhere the hell is the donate button? People that get this thing going should send some kinda fiat or coin your way. This has tremendous implications. Edit: The readme mentions use of dockers... if my Emby server is on a VM that's on the same network, does that make a difference (meaning can I access the mounts that this script uses)?

I disabled IOMMU in the CPU settings and that seems to allow the machine to boot natively off the USB, with a LSI card passthru no issues. Going to be testing some more in terms of performance, but boot is MUCH snappier than the old (BIOS based) VM booting.

I was able to do this about a week ago... My CPU was unsupported to so I had to add the allowLegacyCPU option to the boot config file AND each VM's advanced configuration parameter needs an monitor.allowLegacyCPU = True That leads to SUPER slow boot times on older VMS and even VMs upgraded to 7.0. however, newly created VMs are fine. I'm trying to do USB boot and that works, but for some reason, USB passthrough with my SAS card does NOT work. Unraid seems to find the devices ... but right before the login prompt crashes. I can't seem to get any info before the crash happens. Anyone else having trouble with passthrough on Esxi 7.0 and UnRaid? The reddit thread linked above seems to be able to even see the card as passthrough capable, which I don't think applies to my situation.

Thanks for getting back to me. I ended up sticking with Unraid as a VM inside ESXi for the time being.

did any of you guys have a permissions issue starting the VM? I kept getting a Permission Denied error trying to start the VM...

do you remember what you did to get this fixed? my VMs are all topping out at about 70Mbit ... even though I have 400Mbit. If i boot Unraid with GUI, i can get about 360Mbit, which is what I used to get in my VMs (same hardware) on ESXi. Even creating a new VM doesn't seem to help.

Thanks - the parity sync is still going, found 792 corrections - probably the highest I've ever seen. Then again, haven't really lost power in the middle of a known write-heavy operation. Love how UnRaid keeps on truckin! I've lived in this house over 5 years now... only lost power 3 times ... Sandy, another major unnamed storm, and of course yesterday. Had to be in the middle of this. Oh well.

ooyy!! I was in the middle of the mirror method (https://lime-technology.com/wiki/File_System_Conversion#Mirroring_procedure_to_convert_drives) - but in the middle of step 8 we had a power outage (longer than my UPS can support). After the boot up - I let it run through the parity check (still going). Can I restart step 8 (rsync -avPX /mnt/disk10/ /mnt/disk11/) ? Or do I need to reformat the destination disk and start it over?

Thanks - that's what i'm going to do then!

Yeah - just might do that - after my parity check finishes, if nothing changed.

That's a relief! (I think). Hopefully it stays put.

Yeah, a second reboot cleared all that up - don't know what happened first time i've seen it on all drives. I'm running a Parity check w. corrections fingers crossed.

Thanks - sorry I updated (unraid was 6.4.0 to latest ) and rebooted before I saw your post... but here's where its weird - all my drives came back with high CRC errors. So perhaps i've got to go jiggle my SAS expander's cable.

Thanks - to clarify ... i was copying from Disk 1(3tb ReiserFS) to Disk 14 (3 TB xfs-enc). Disk 14 ran out of space. Disk 13 is a 4tb (xfs-enc) . I'd like to completely abort DISK 1 to Disk 14 and just go from Disk 1 to Disk 13. At the end of that, I'd like to format Disk 13. Then again - what's the point...I guess I should just delete the files there. For some reason after this copy round ended, UnRaid seems to be slow/unresponsive. I'm going to restart the array hopefully that helps

Okay - so using the Mirror method, I was copying from a (nearly full) 3TB reiserfs drive to a 3TB xfs-enc drive. .... and it ran out of space at the end of the copy (It was copying the last 20gb file). I have a 4TB drive that I can move this stuff on to. What's the best way to format the 3TB destination drive? Just swap file systems again? Is there an easier/better way? I can skip this drive until I get to the 2tb reiserfs drives.

Thanks Gary - I was thinking of doing offline copy to my Media PC as well, but not looking forward to doing it over the network. I'm following the steps of the Mirror Method (the one that @Frank1940 linked). Thanks @trurl - I do care where data goes for two drive slots as I use the NFS exports ... but do like having the drives map physically back to their physical slot in the case also. Thanks @Frank1940 - that's the process I'm using - and mapped everything out on Excel so i know what's going where... what to physically move when done, and what the end result is. I think the step that got me hung up is sort of a "test" that the array starts up? I thought for some reason the array wouldn't start until the drives were formatted or matched the format. The steps do say to go back and set them properly, but I think I just got nervous knowingly assigning the wrong format.

I just realized I don't understand Step 15.. Important! Click on each drive name (e.g. Disk 10 and Disk 11) and swap the file system format of the drive - if it's ReiserFS change it to XFS, if it's XFS change it to ReiserFS; it's important to swap the disk formats as well as the physical drive assignments Wouldn't changing this on the target drive require me to format it? So in my case I went from Disk 8 to Disk 13. Disk 13 was an empty drive that was Formated xfs-enc. I finished moving everything over. Did new config, and assigned the new physical disk to Disk 8, and the old disk to slot 13. I understand changing Disk 13 here will be desirable (so that the next step can start with a blank disk in the right format), BUT wouldn't changing the new disks format (now in disk 8 ) require format?

Gary - the move to the newer drives is mostly for space/performance reasons. The older ones were "green" 2TB drives; "newer" are HGST NAS drives, about 2 years old, were pre-cleared and in my test array. I'm planning on using dual parity, but not until all of this is done. I just want the Drive # to match the physical location on the case, cos OCD, perhaps. Thanks

Admittedly, I have not read all 33 pages of this. I did read the first few maybe a year or so ago. I think I understand the process pretty well. I'm actually injecting some larger drives into the array, as well as making sure certain important files go on these newer drives. I've mapped it out so that the older drives get pulled off to the side and will be converted last. Here's my conundrum... I'd like to keep my physical slots the same in my case. So after step 17, I'm thinking of physically swapping the slot. So in essence my copy to drive will always be in the same slot (and will assign it Disk 13). Or is it better to keep walking through the array and then move the physical stuff at the end?