Data juggling between drives

[Sandwich]

I have about 5.75Tb of data that I'm trying to transfer from my Drobo to an unRAID box. I've rsync'ed all the data over to the unRAID box as it's currently set up (see below), and am running a parity check:

 

Current setup:

• unRAID
  • 8Tb (Parity)
  • 6Tb (Data; with 5.25Tb of the data)
  • 2Tb (Data; with 0.5Tb of the data)
• Drobo
  • 4 × 3Tb (5.75 of data with single-drive redundancy)

 

My understanding is that once the parity check is complete, my data will be safe on unRAID from a single-drive failure. Then, my goal is to transition to the following:

 

Desired setup:

• unRAID
  • 8Tb (Parity)
  • 6Tb (Parity)
  • 4 × 3Tb (Data)
  • 2Tb (Data)
• Drobo (retired)

 

Of particular note is that the 6Tb drive will need to transition from being a data drive to being a parity drive. To that end, I'll need to move the data from it to the other data drives (the 3Tb drives coming from the Drobo). But what's the best way to go about doing this? Do I pull the drives out of the Drobo and put them into the unRAID box one at a time, rebuilding parity each time, or add them in all at once? And particularly, once they've been added, how do I move the data from the 6Tb to be spread out among the 3Tb drives?

 

Finally, much of this data is related to my Plex Media Server (PMS) and Winamp libraries. I'd like to get those up and running with the data in unRAID (PMS in a Docker and Winamp on my PC) before I pass the point of no return in removing drives from the Drobo. Winamp I'm not too worried about, but if I get PMS set up while the data is on the 6Tb and 2Tb drives, will I have to update things once the data gets moved to the 3Tb drives? Like, if PMS is set up in a Docker, where are those PMS/Docker files stored? Will that need to move too?

 

Thanks for any help!

Edited April 8, 2021 by Sandwich
Formatting tweaks

[Sandwich]

Nobody?

[SimonF]

The Unbalance plugin will allow you to move data from the 6TB drive to others. As long as the same shares are used for PMS you will not need to change when you move the files as will still be in the same share.

[Sandwich]

Ahhh, thanks! So I shouldn't be doing any data relocation from the terminal then?

Edited April 9, 2021 by Sandwich

[ChatNoir]

39 minutes ago, Sandwich said:

Ahhh, thanks! So I shouldn't be doing any data relocation from the terminal then?

You can if you prefer and you are sure not to make a mistake and loose files but you don't have to.

[SimonF]

56 minutes ago, Sandwich said:

Ahhh, thanks! So I shouldn't be doing any data relocation from the terminal then?

As ChatNoir said you can and say use mc, Unbalance allows you to move from one disk to many or from many to one.

 

[Sandwich]

Awesome, that looks like it'll do the trick, thanks so much ChatNoir and SimonF.

 

That leaves two questions - the one about transferring the four 3Tb drives all at once or one at a time, and the one about if moving data around drives will disrupt PMS... EDIT: Specifically, I'm thinking more of Plex's internal cache & data files, not of the audio and video media files that it manages.

Edited April 9, 2021 by Sandwich

[ChatNoir]

44 minutes ago, Sandwich said:

transferring the four 3Tb drives all at once or one at a time

If the data has been transferred from the Drobo to the Unraid Array, I do not see a reason to add the disks in several steps.

I would just:

• shut down both devices
• transfer the drives
• Start Unraid
• Stop the Array
• Add the 4 3TB drives
• Start the Array, Unraid will ask to format the drives. CHECK that the drives in the list are the correct ones.
• Move the data around with Unbalance

Then, I am not 100% sure, I am still not super experienced with Unraid.

I'd say look at the Shrink Array procedure.

I am thinking the second method, then add the 6TB drive and sync it.

 

54 minutes ago, Sandwich said:

if moving data around drives will disrupt PMS... EDIT: Specifically, I'm thinking more of Plex's internal cache & data files, not of the audio and video media files that it manages.

It would probably not be super efficient, specially since your metadata files are on a spinner, not a SSD.

In any case, I would not advise to move the docker stuff with the docker service running.

 

I'd say :

• Unbalance every thing except appdata and system shares
• Settings / disable Docker and VM services
• Unbalance the remaining shares (will take a long time with many small files)
• Enable the services

[Sandwich]

On 4/9/2021 at 2:58 PM, ChatNoir said:

It would probably not be super efficient, specially since your metadata files are on a spinner, not a SSD.

 

Hmm... I do have a 1Tb SSD I could add. Would it make sense to do something like partitioning part of it (half?) to be a cache drive, and leave the other part for holding appdata or whatever?

 

  

On 4/9/2021 at 2:58 PM, ChatNoir said:

In any case, I would not advise to move the docker stuff with the docker service running.

Ooh, so if I'm messing around with the PMS docker container, just stopping PMS isn't enough; I should halt the entire docker service?

[ChatNoir]

1 hour ago, Sandwich said:

Hmm... I do have a 1Tb SSD I could add. Would it make sense to do something like partitioning part of it (half?) to be a cache drive, and leave the other part for holding appdata or whatever?

No need to partition, a single disk can manage both. Set appdata and system to Prefer:cache and the share you want to cache for data transfer to Yes:cache.

 

For example, I only use a single disk pool for this.

 

1 hour ago, Sandwich said:

Ooh, so if I'm messing around with the PMS docker container, just stopping PMS isn't enough; I should halt the entire docker service?

Depends on what you do. If you just edit the docker, that's fine. If you want to move the docker image (system) or your appdata, it is advised to disable the service entirely.