Coming From FlexRAID


Recommended Posts

Just had a few questions before I make the decision to purchase.

 

I am coming from a simple media server setup. I used my system for streaming music and video to Zidoo and Dune devices throughout my home. The computer is running Windows 7 Pro via a Shuttle SH67H3 with an i5, 16GB of RAM, integrated graphics, Asus BLU-RAY writer.... the PC is connected to a 4U 45-Bay SAS2 SATA SC847E26-RJBOD via a Dell H200E 6Gbps SAS HBA Dual-Port External Controller. The drive pool and parity are controlled by FlexRAID.

 

One of my 2TB HGST drives, after 12 years, has failed.  And since FlexRAID support no longer exists, I "hoping" to replace and rebuild the drive with a new one with FlexRAID, before I make the move to Unraid.

 

My requirements are pretty simple I think... At the very minimum, I need to be able to use my Shuttle (in some form) to run Windows 7 as I have before. I'd still need to use my existing BLU-RAY drive to rip (MakeMKV and ClownBD) BLU-RAYs and UHDs and save them to the drive pool of the external JBOD.

 

Would I just be installing Windows 7 as a VM and controlling it from another PC via RDP? Would the VM have access to the BLU-RAY drive? Or is there a simpler solution?

 

Thanks so much,

John

 

Edited by Litlgi
Link to comment

Unriad would become the main function of the shuttle case PC.  You would then run VMs/Dockers for the various functions you wish.  And example would be there is a docker for makemkv, which should allow for the ripping of your physical library.  You can also pass various things to VMs (Don't have a BLU-Ray player in my system to know what access VM's/Dockers have to it).  Good luck with the flexRAID rebuild never had one work, where as unraid I've had a couple work as I replaced smaller drivers with bigger drivers, it's slow on my system but it has worked every time.  What I would do is find a spare machine build it up with the bare min and see if it will do what you want.  There is a Full 30-Day trial that will get you started, you can then move the USB key to the other system attached the drives and it will continue to work in most cases as if nothing had happened.

Link to comment

Thanks for the quick reply... That's already a good start considering the lack luster and now non-existent support if the flexraid community.

 

I've ordered a bunch of drives to copy my media to the new unraid system... I'm a little fearful of wiping out my old drives without a backup.

 

I guess I have to look into what dockers and vms can do... 

 

It's probably safe to assume that if Windows can run the software... It should be no problem with it running in VM... such as Ipvanish, uTorrent and JDownloader.?

 

All of my RAID drives (20) are installed in JBOD enclosure.... So I only have one way of connecting to them... And that is via the Shuttle. So s second "test" system really isn't possible right now.

 

Thanks again.

Link to comment
On 1/5/2020 at 10:11 PM, Litlgi said:

the PC is connected to a 4U 45-Bay SAS2 SATA SC847E26-RJBOD via a Dell H200E 6Gbps SAS HBA Dual-Port External Controller. The drive pool and parity are controlled by FlexRAID.

 

Would I just be installing Windows 7 as a VM and controlling it from another PC via RDP? Would the VM have access to the BLU-RAY drive? Or is there a simpler solution?

 

On 1/6/2020 at 12:30 AM, Litlgi said:

I guess I have to look into what dockers and vms can do... 

 

It's probably safe to assume that if Windows can run the software... It should be no problem with it running in VM... such as Ipvanish, uTorrent and JDownloader.?

 

All of my RAID drives (20) are installed in JBOD enclosure.... So I only have one way of connecting to them... And that is via the Shuttle. So s second "test" system really isn't possible right now.

 

The Dell H200E looks to be a LSI clone so you should be able to access your JBOD enclosure in Unraid just like it's directly connected internally. I still recommend you read up on flashing HBA controllers to IT mode since it's generally recommended on the forum.

 

You can run a Windows VM and access via RDP (that is a pretty common use case).

Ripping your physical media should be done in docker (e.g. ripper or makemkv dockers - I have used both and they are both good) - it's a lot simpler to set up.

Passing through an optical drive to the VM is complicated and writing optical discs does not work - at least I have never managed to make it work.

 

If you can run something in Windows baremetal, it's 99.99% sure to run in a Windows VM. The only 2 exceptions are: nested virtualisation (i.e. a VM under another VM - it needs hardware support) and software that is purposely coded to NOT work in a virtualised envi (e.g. *cough* nvidia *cough*)

But just as ripping physical media, anything that has an Unraid docker should be run as a docker. It's safer and more efficient.

 

A docker for Unraid is like an app on your Android phone. There are many dockers on Community Apps so I'm sure you will find something useful.

For example, there are dockers that allow Transmission or Deluge to run through a VPN (any VPN that supports OpenVPN should work but PIA is easier to setup). The dockers also have Privoxy setup so you can use the Privoxy proxy for jDownloader docker, which allows it to run through the same VPN.

Link to comment

Thank you very much for the thoughtful reply...

 

5 hours ago, testdasi said:

The Dell H200E looks to be a LSI clone so you should be able to access your JBOD enclosure in Unraid just like it's directly connected internally. I still recommend you read up on flashing HBA controllers to IT mode since it's generally recommended on the forum.

I did put Unraid on a thumb drive... When the OS booted... all my drives in my JBOD enclosure were recognized by Unraid via the Dell card. So maybe I don't have to flash the card? I think it is already in IT mode.

 

5 hours ago, testdasi said:

You can run a Windows VM and access via RDP (that is a pretty common use case).

Ripping your physical media should be done in docker (e.g. ripper or makemkv dockers - I have used both and they are both good) - it's a lot simpler to set up.

Passing through an optical drive to the VM is complicated and writing optical discs does not work - at least I have never managed to make it work.

Would connecting the Blu-ray to the Shuttle PC via USB make the it easier to access in the Windows VM? Writing is not really important to me?

 

I've been reading about preclear... Is it possible and advisable to clear multiple drives at once to speed up the process time?

 

What would be the best and most efficient way to copy data from my 20+ Flexraid drives to Unraid drives? 

 

Thanks again.

Link to comment
40 minutes ago, Litlgi said:

I did put Unraid on a thumb drive... When the OS booted... all my drives in my JBOD enclosure were recognized by Unraid via the Dell card. So maybe I don't have to flash the card? I think it is already in IT mode.

 

Would connecting the Blu-ray to the Shuttle PC via USB make the it easier to access in the Windows VM? Writing is not really important to me?

 

I've been reading about preclear... Is it possible and advisable to clear multiple drives at once to speed up the process time?

 

What would be the best and most efficient way to copy data from my 20+ Flexraid drives to Unraid drives? 

If it's already in IT mode and Unraid detected drives fine then nothing else is needed.

 

Connecting via USB will work (e.g. passing a USB device to a VM) but then if writing is not important then you can simply use dockers.

 

I have little experience with FlexRAID so not too sure on most efficient migration.

Link to comment

I actually came from FlexRAID myself, several years ago at this point. (And am very happy with the decision. unRAID isn't perfect - very little is - but it's vastly superior the FlexRAID.)

 

I can't remember off the top of my head, but I'm pretty sure it allows for drive removal? I know you could expand a drive at a time, so it makes sense you could shrink it a drive at a time as well. If you have enough spare slots in your Shuttle/Netapp box, then you can copy data to unRAID with a starter disk, take that drive out of Flex, add to unRAID, copy the next, remove, add, copy, and so on.

 

If you want to be secure about it, add the parity at the start, if you want it to be faster (it'll be very slow if it has to calculate parity during the move) you can leave adding the parity disk until the end.

Link to comment

Thanks for the reply... Glad to hear you are happy with the switch to Unraid.

 

3 hours ago, -Daedalus said:

I can't remember off the top of my head, but I'm pretty sure it allows for drive removal? I know you could expand a drive at a time, so it makes sense you could shrink it a drive at a time as well. If you have enough spare slots in your Shuttle/Netapp box, then you can copy data to unRAID with a starter disk, take that drive out of Flex, add to unRAID, copy the next, remove, add, copy, and so on.

 

I know you can remove drives from the FlexRAID drive pool... But since the drive pool will no longer be accessible in Unraid... And I only have one PC to connect to expander... I'm not sure it matters. Or are you saying there is a way to run Flexraid and Unraid on the same machine at the same time?

3 hours ago, -Daedalus said:

If you want to be secure about it, add the parity at the start, if you want it to be faster (it'll be very slow if it has to calculate parity during the move) you can leave adding the parity disk until the end.

I have about 90TBs of data to move... It's going to be a very, very long process, so I think I'll hold off on starting with parity in the beginning... I plan on keep my old drives as backup.

 

Do you know if I can preclear more than one drive at a time... Or if it's advisable to do so?

 

Thanks again

Link to comment

If I'm remembering correctly, the individual drives on FlexRAID still have a filesystem on them. Can data be accessed on individual drives? If yes, then you can use the Unassigned Devices plugin(1) to mount one drive at a time in unRAID, and transfer the data across that way.

 

If you have enough physical space for the drives, another option might be to install unRAID, assign you new disks to the array, then load your existing machine as a VM. You could pass all your FlexRAID disks to it, get it started, then transfer the data over the network. I think virtio is smart enough to keep the transfer internal to the box too.

 

You absolutely can preclear more than one drive at once. Depending on the number, you might be bottlenecked by your controller/chipset, but so far as I know, there isn't a limit.

 

(1) It's kinda what it sounds like; it lets you make non-unRAID drives accessible on your unRAID box. Very handy if you need to copy data from an NFTS disk from a Windows machine, for example.

Link to comment

You are kind of touching what I fear about my FlexRAID setup... I am currently using folder priority... But I am not sure how deep into the folders FlexRAID goes with storing important folder info or files on another drive. So my fears is that I won't be able to transfer folders and files from one FlexRAID disk to an Unraid disk without potentially missing something during the copy and paste procedure.

 

Would I be able to somehow add my existing Windows 7 OS  (which has FlexRAID installed) as a VM in Unraid... so that I can start transferring files from the FlexRAID drive pool to the drives to be used in Unraid? Unfortunately... The FlexRAID drives need to stay in the JBOD expander. Which is where I would be adding the new Unraid drives. I have no way of having two separate systems... As I only have access to one JBOD expander.

 

Thanks

Edited by Litlgi
Link to comment

Perhaps before complicating the matter with running FlexRAID in a Windows VM (which is rather complicated since you can't pass through the HBA to the VM due to Unraid needing it to access the JBOD), have you tried just loading Unraid + Unassigned Devices plugin to see if it can actually mount the FlexRAID drives individually?

Then if the drives can be mounted, perhaps copy a few of those files over to see if they are valid files.

 

I vaguely remember FlexRAID to be what Daedalus said (i.e. each disk has its own file system) but it has been too long.

 

Also, you can use Trial license to test it out before purchasing. Since your migration process is rather massive, I'm sure the LT guys will be understanding enough to give you an extension to the Trial so you can test the process out properly.

Link to comment

I came from flexraid like a month ago.  I basically had two extra disks created my unraid server with parity so that I was not putting my data at risk and then started transferring my data over one drive at a time.  Once the old disk was emptied I precleaned it and then added it to the unraid settings and started the next disk.  I used the unassigned disk plug-in to mount the drives as they where ntfs it was no problem (Which I believe your system will be as well).  And I also precleaned more than one disk at a time with out any problem.  I'm not sure it's any faster, as it has to split bandwidth between your drives.  This of course is my system yours sounds like it much better than mine :(..   

Edited by Aaron Arnold
Link to comment

Thanks Aaron...

 

Were you concerned with... and how did you overcome the issue with files and folders (of one movie or TV show) potentially being on separate drives in flexraid... and not having a complete file structure when transferring one drive at a time to unraid?

 

My new drives will be here tomorrow... I'm going to try to rebuild one failed drive in flexraid before attempting anything with unraid.

Edited by Litlgi
Link to comment

I just want to say i also came from Flexraid and i must say Unraid have been great very easy to setup.

The dockers are amazing works well.

The community at Unraid is very very helpful and that made my purchase great.

 

If you need some help on the setup have a look at this 

https://www.youtube.com/channel/UCZDfnUn74N0WeAPvMqTOrtA

Spaceinvader one Vids are amazing.

 

Lost a drive the other day and was so easy to replace and rebuild.

 

The only problem i do have with Unraid is that if you do a parity Check it take very long and if that runs you cannot really use the machine.

And i also noticed if you copy Data from your Cache drive to your Shares it also take a long time.

 

But apart from that it is very Good and i am very happy.

 

 

 

 

Link to comment
9 hours ago, Litlgi said:

Thanks Aaron...

 

Were you concerned with... and how did you overcome the issue with files and folders (of one movie or TV show) potentially being on separate drives in flexraid... and not having a complete file structure when transferring one drive at a time to unraid?

 

My new drives will be here tomorrow... I'm going to try to rebuild one failed drive in flexraid before attempting anything with unraid.

Nothing you can do if migrating drive-by-drive. If you are really OCD, you have to manually move things around AFTER migration.

Link to comment

Maybe another option?

 

As I mentioned earlier... I'm concerned about the file structure of my files and folders on my flexraid.... 

 

Would it be possible for me to leave my Shuttle and JBOD Expander running flexraid... While transferring my files and folders over the network to an older PC running unraid? Although doing it this way, I would be limited to filling up one drive at a time. Then disconnecting it adding another oh, and so on. When finished, would I then be able to take those filled up unraid drives and plug them back into the shuttle system attached to the jbod expander running unraid?

 

PS. Currently rebuilding the sale drive in flexraid... Hopefully that completes without a problem.

 

Thanks again

Edited by Litlgi
Link to comment

I replicated my shares from the flexraid setup and just moved everything over on drive at a time, the only thing that is duplicated is the folder structure so it's not a big deal to continue over writing it as it just doesn't do anything.  I used the krusader docker to do the moving.  you could even create the folder structure a head of time using the linux commands without moving anything (Which is what I did for testing to see what it would actually look like with out moving my data, and sorry I don't remember what the command was that I used.  But a quick google will work on finding the information).    Keep in mind that you would create the root folders, and then copy the folders structure of each root folder.

 

An example I had a root folder pictures so I create a share called pictures and then used krusader to copy/move pictures from drive 1 to the unraid shares drive, it will warn you that it's over writing pictures, you say yes and then it continues.  Disk 2 you would do the same but this time you will get a second warning as disk 1  create the folder structure already, you will also say over write, and so on.  

 

The data that is already there will remain and the new data will also be written.  You can test this if you wish with a small subset of drives using the onboard controller of you motherboard.  I understand your concerns and the only way that you can be sure and put your mind at ease is to test it.  Which is what I would suggest for anyone who is thinking of going down this road.  I used some small disks and just played until I was sure/happy that it was the right way to do it.  I just expanded my test box to my production box, this also allowed me to test drive replacing and recovery of disk function, which worked very well.  As unraid is very flexible in this case it seems to handle hardware changes very well, there are outlining case (From what I've read) that don't always work like anything nothing is perfect.    Hope this is helping/helped :).

 

 

 

  

  • Thanks 1
Link to comment

Well... My FlexRAID restore failed. I lost 78 Blu-ray folders and a bunch of other data. But since I own the Blu-ray disks... It's not a complete loss.

 

Preclear is working well... I am limited to two drives at a time. Anymore than that, the HDD temps go into the warning zone. My JBOD expander fans are capable of cooling the drives to less than 30 degrees Celsius... But the noise from the fans at 6500 rpm is unbearable. I am using a Supermicro CSE-PTJBOD-CB3 controller which has fan control... But the auto fan plugin does not recognize it. Since I will be using Unraid as a media server... It will be unusual for more the one HDD to be active. So the fans should have no problem with the cooling at a less noisy 2500 rpm. But if anyone can point me in the direction of a script to help with fan control... I'd appreciate it.

 

I have a have a week or so more of preclear time before I can start transferring data.

 

Question... Unraid tier 3 says it's capable of unlimited attached storage devices... But in the array device section of the Main tab... The drop down menu of slots only goes to 30. I need 45. How do I fix this?

 

Thanks

Link to comment
37 minutes ago, Litlgi said:

Well... My FlexRAID restore failed. I lost 78 Blu-ray folders and a bunch of other data. But since I own the Blu-ray disks... It's not a complete loss.

 

Preclear is working well... I am limited to two drives at a time. Anymore than that, the HDD temps go into the warning zone. My JBOD expander fans are capable of cooling the drives to less than 30 degrees Celsius... But the noise from the fans at 6500 rpm is unbearable. I am using a Supermicro CSE-PTJBOD-CB3 controller which has fan control... But the auto fan plugin does not recognize it. Since I will be using Unraid as a media server... It will be unusual for more the one HDD to be active. So the fans should have no problem with the cooling at a less noisy 2500 rpm. But if anyone can point me in the direction of a script to help with fan control... I'd appreciate it.

 

I have a have a week or so more of preclear time before I can start transferring data.

 

Question... Unraid tier 3 says it's capable of unlimited attached storage devices... But in the array device section of the Main tab... The drop down menu of slots only goes to 30. I need 45. How do I fix this?

 

Thanks

Are you using the Preclear queue to manage your preclear sessions?

Link to comment
2 minutes ago, Litlgi said:

Yes I am... I have it setup so that I am preclearing drives that are at opposite ends of the backplane.... Seems to be working.

 

Any idea on how to increase the number of slots from 30 to 45?

 

Thanks

This is not possible at this time. Like it said, unlimited attached storage, not unlimited devices in your array.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.