Some questions for my first setup - Migration from MS Server 2019

[Maddeen]

Hi there,

I think about building my own NAS based on unRAID. 

currently I‘m using a windows server 2019 (i7 3770k; 16GB RAM) with three independent drives:

a) 256GB SSD (OS)

b) 4 TB HDD (Shares)

c) 2 TB HDD (Backup for some important files stored on the 4TB drive. ) 

 

First question is about my contemplated migration process:

Hint: I‘ve got another unused 2TB HDD drive for the following task. 

- install the unused 2TB HDD to the server. 

- install unRAID on USB flash drive

- Setting up the Array based on the 2x 2TB HDD, so that I got a total array space of 4TB

- setting up the SSD as cache drive

- copy the data from the 4TB HDD (NTFS formatted) to the array with the help of an external 3,5“ USB Case/dock. 
(Is this the best practice? Or can I leave the HDD internal attached (SATA) and copy the data too?)

- as soon as all my data is copied successful, I can format the 4TB HDD drive and set it as the parity drive??

 

 It’s this a solution or a possible way to migrate without loosing the data stored on the 4TB HDD?

Sadly I don’t have another 4TB drive to make it even easier 🙈

 

And please let me ask a probably stupid question. But on every guide referenced to unRAID I found the information that the parity drive must be as big as the biggest „data“ drive. 
That’s understandable. But in all of this guides there a 2 or even more drives with the same space. For example:

1x 4TB parity drive 

3x 4TB data drive (12TB Array Space)

 

And that’s the point where my brain got a kernel panic. 
How can a 4TB parity drive store all necessary data for 12TB data, when it comes to a drive failure and the necessary rebuild??

 

 Thanks for anybody being my light in the dark. 
 

P.S English ist not my native language. I apologize for any stupid grammar. 

 

Edited May 1, 2020 by Maddeen

[trurl]

2 hours ago, Maddeen said:

leave the HDD internal attached (SATA)

This ^

 

2 hours ago, Maddeen said:

format the 4TB HDD drive and set it as the parity drive??

Parity has no filesystem so no format. Just assign and let parity build. 

 

2 hours ago, Maddeen said:

How can a 4TB parity drive store all necessary data for 12TB data, when it comes to a drive failure and the necessary rebuild??

Parity doesn't have any data. Parity is basically the same concept wherever it is used in computers and communications. Parity is just an extra bit that allows a missing bit to be calculated from all the other bits. To rebuild a disk requires parity PLUS ALL the other disks.

[Maddeen]

@trurl thanks for you fast reply. So no problem building the parity after the copy job is done, right?

I'll try this later on the day and give a feedback.

 

Have a nice day

[trurl]

7 hours ago, Maddeen said:

So no problem building the parity after the copy job is done, right?

Right.

 

But I always have to add.

 

Parity, whether RAID or Unraid or whatever, is NOT a substitute for backups. You must always have another copy of anything important and irreplaceable. Plenty of more common ways to lose data besides disk failure, including simple user error.

[Maddeen]

@trurl - thanks again.
All my irreplaceable files are spread out and synced to multiple cloud services.

But - as soon as I got more budget - i'll build a second unraid server for additional backup.

 

For now - honestly I'm still at the beginning - everything works fine.
I had set up the users, the shares, the array, installed some recommended plugins and "krusader" docker (changed PUID to 0 for root access) for starting the copy job.

Let me ask another question - because I'm pretty sure that I read a important note when gathering information about unraid server.
But I'm not able to find that note again. If I remembered right, it was something like

Quote

be sure to copy your data to ?shares/discs?- otherwise you'll loose your data

Seemed to be a problem with the authorization/access/permissions or something like that

So what's the correct destination when making internal copy jobs? To shares? or directly to the disk?

 

Thanks again- habe a nice sunday.

 

Edited May 3, 2020 by Maddeen

[trurl]

Since that quote doesn't actually reference another post, i can only guess it is a paraphrase that lost the actual meaning, or the original poster was confused or not expressing the meaning well. 

 

I think perhaps it is referring to the problem that can arise if you mix user shares and assigned disks when moving/copying. Linux doesn't know that the user shares and the assigned disks are just different views of the same files and so can try to overwrite what it is trying to read if the paths work out that way. 

 

Since you will be copying from an unassigned device there is no problem whether the destination is a user share or an assigned disk.

[Maddeen]

Ok - fine - thanks again.. and -- surprisingly - you was "the guy with the note" I found it here:

 

Quote

Be very sure you move from disk to disk and not user share to disk or you could get the User Share Copy Bug

 

[trurl]

You should have followed the link I gave in that old post and it would have explained the problem. 

[trurl]

Or maybe you did follow it and the "quote" you gave was an attempt to paraphrase your understanding of it.

 

Do you understand the explanation I just gave above? 

[Maddeen]

Jep - I understand it now. So I’ll safe this until the first time I have to change drives 😜 THX

[Maddeen]

Update: Thanks to your support @trurl I'll definitely buy unraid - I'm nearly finished with the basic setup and very happy. 

• Array set up (1x 4TB, 1x 2TB, 1x 1TB and a 4TB Parity disk) - check
• Cache drive (Intel 256GB) set up - check
• Installed recommended plugins - check
• Shares set up - check
• Plex is up and running - check 

But now it comes to some (for me ) fine tuning or deeper questions, where maybe someone can help me out
Hope that is ok to gather all those questions here in my thread instead of making x threads with one question.

1) Plugin FixCommonProblems tell me that theres a problem mounting an unassigended drive to binhex-krusader without using SLAVE setting.

But - after copying all my data - I don't have any unassigended drives - all assigned.
And I cant found any information about "mounting mode". I remember, that I set (where ever) it days ago - but where?

 

2) Strangely I have two "internal" user-shares. Not sure if this is the correct wording, but I have a "user" and a "user0" folder.

Searching via google indicates that this is normal when you have more than one entitled user in the SMB-permissions.

But I haven't. All my shares just have one SMB permission - myself

How can I get rid of either "user" or "user0" because that confuses me - especially when making internal references for example in "CA Backup / Restore Appdata" --- i'm not sure what's the correct folder. /mnt/user/xxxx or /mnt/user0/xxxx

And both have different folders in it. 🙈

IMHO the folder "user" is the old one - because it got the folders "appdata, domains, system" from some days ago before I added a cache drive - but i'm not sure! 

Now all this folders/files are stored to the cache drive and set it to "only" (Option = Use Cache Drive) - and it seems to be consistent because in the share overview those folders (appdata, domains, systems) got the orange triangle telling me, that they are not protected by the parity drive

3) I got a error message due to the .TRASH folder. I deleted it - so everything fine, i think. But what does this folder do? I searched for a trash setting or a button where I can restore things that were accidentally deleted and moved to the trash-folder. But I can't found one - so whats the use for this folder?

4) Last but not least - as I said I moved all necessary files (appdata, domains, system and the docker.img) to the cache drive for better performance. 
Just to validate, I will list all my actions to make sure, that I made it correctly and as "best practice"
- Share appdata/domains and system moved to cache drive and set the option "use cache drive" to only

- In Settings/Docker I set up vDisk location to /mnt/cache/system/docker/docker.img and default appdata storage to /mnt/user/appdata.
I was not able to change it to the cache directly because I cant even navigate out of the user home directory.
Is this correct? Maybe it is because I configured "appdata" to be stored only on the cache drive? But as you can see, its in the other "user"-share and not "user0"-share (Reference to Bullet #2 - why two folders user and user0)

- As guided in another thread I deinstalled my two dockers, move the docker.img/appdata and the reinstalled it. Also set the mappings for both docker-apps (krusader/plex) for parameter /config to /mnt/cache/appdata/

I think that's it for the moment - or for the thread topic related to a first setup.
For any further - and even more detailed questions, I'll set up different threads in the specific subforums. 

 

Thanks a lot - have a nice friday.

Edited May 8, 2020 by Maddeen

[JonathanM]

The biggest thing that needs to be addressed is your understanding of unraid's folder system.

 

Unraid has several ways to view or work with your data, depending on the circumstance.

 

<Direct disk access>

/mnt/disk1-disk28 are your parity protected array disks

/mnt/cache is the cache pool, regardless of how many disk members it has

 

<User shares>

/mnt/user is the root folders of BOTH the /mnt/diskX tree AND /mnt/cache

/mnt/user0 is ONLY the root folders of /mnt/diskX, it omits the /mnt/cache portion

 

There are no extra copies shown, if you delete something in one tree it's gone from all of them. Because the same exact content can show up in multiple trees, it's vital that you not copy or move between the disks and user shares unless you understand exactly where and what is happening.

 

Directly referencing the disks via /mnt/diskX and /mnt/cache is not preferred, and when everything is working the way Limetech intended is not needed. However, there are still some situations where the preferred all encompassing /mnt/user tree doesn't quite work as intended, and using the disk paths is needed for some applications to work correctly.

[Squid]

3 hours ago, Maddeen said:

And I cant found any information about "mounting mode". I remember, that I set (where ever) it days ago - but where?

 

https://forums.unraid.net/topic/57181-docker-faq/#comment-564360

3 hours ago, Maddeen said:

3) I got a error message due to the .TRASH folder. I deleted it - so everything fine, i think. But what does this folder do? I searched for a trash setting or a button where I can restore things that were accidentally deleted and moved to the trash-folder. But I can't found one - so whats the use for this folder?

Krusader probably created it when you deleted some files.

[trurl]

4 hours ago, Maddeen said:

Last but not least - as I said I moved all necessary files (appdata, domains, system and the docker.img) to the cache drive for better performance. 
Just to validate

Looks like your questions got answered, but to validate your shares are using the intended disks, go to Shares - User Shares and click the Compute All button. You may have to wait a few minutes for the results. Refresh the page if results don't appear after a while.

[Maddeen]

Thanks for all your replies. I'll go trough my bullet points for better overview

 

#1 - @Squid - Thanks - that solved the question "where was this setting". But that's not solving my problem.

Because I dont have any unassigned disks now to change any setting. 
That's why I dont understand the Error-Message either.

How it can be possible that krusader got a mounted unassigned disk that doesnt even exist?

Here's a screen -- there are definitely no unassigned disks

 

#2 - @jonathanm - Thanks. So the info that i saw on a youtube video was just wrong and led me to the wrong way.

With your explanation --> user = root folders + cache | user0 = root folders only it makes sense and is fine for me. 

So - in other words - if I want to link to a root folder, there is no difference between /user or /user0. 

But as soon as I want to link to a cache directory, I must use /user because otherwise there is no folder  

Or to even make it easier - for internal settings within unraid its the best to ever use the /user because it always represents all available folders.

5 hours ago, jonathanm said:

Directly referencing the disks via /mnt/diskX and /mnt/cache is not preferred,

I use this reference at the following points - see Screenshots 1-3

- Docker hosted application mappings like Plex and Krusader (Screen 1)

- Docker vDisk location (Screen 2)

- CA AppData backup plugin (Screen 3 - copied from "spaceinvader one" youtube tutorial)

 

If I understand you correct, I need to change those references always through the share-tree like /mnt/user/appdata.

And due to the settings of the "appdata" share itself (Cache only!) it's assured, that the data will be stored only on the cache, correct?

          Screen1                                Screen 2                                Screen 3

                      

 

#3 - @Squid sounds good. So I can ignore it, because no benefit if theres no option to recover anything from this folder  

 

#4 - @trurl thanks for the hint. Everything seems to work exact as I configured it. 

 

 

Edited May 8, 2020 by Maddeen

[Squid]

1 minute ago, Maddeen said:

Because I dont have any unassigned disks now to change any setting. 

It's right in your picture.   /unraid_unassigned mapped to /mnt/disks.

[Maddeen]

Ahhh . so it don't point to a specific drive (thats what I thought) -- it's just pointing to a mapping no matter if there are devices or not. 

So the error message (imho) is wrong or misleading. Because there can't be a slave-setting for a drive that doesnt even exist.

The error-message should be like "Krusader got a mapping to an empty disk folder" or something like that. 

 

I thought that this mapping is just a "link" -.- and IF there is any unassigned disk it would be presented there. So I left this mapping. 

But in my case - there was no unassigned disk - so the error message is misleading or wrong.

Because now I need to set up this mapping every time again when I want to browse through a unassigned drive and delete it after.

Otherwise the plugin will remind me again and again ...

 

Forget my explanations@Squid -- like @jonathanm pointed out. I got it now and send you a beer

Edited May 8, 2020 by Maddeen

[JonathanM]

30 minutes ago, Maddeen said:

 

If I understand you correct, I need to change those references always through the share-tree like /mnt/user/appdata.

And due to the settings of the "appdata" share itself (Cache only!) it's assured, that the data will be stored only on the cache, correct?

You don't have to change them if you don't want to, just understand the differences. Setting "Cache Only" only applies to new writes to the user share.

 

Here is an example to hopefully clarify your second quoted sentence.

 

Say TESTING is a user share set cache only. This will create /mnt/cache/TESTING

You, or an app you configured, writes to /mnt/disk1/TESTING/blah.txt instead of /mnt/user/TESTING/blah.txt. That file will be available at /mnt/user/TESTING/blah.txt. Now, even though the share is set cache only, that file (blah.txt) exists on disk1. FCP will complain, and you either will have to manually fix it by moving the file from /mnt/disk1/TESTING to /mnt/cache/TESTING, or set the share to cache prefer and run the mover while nothing has blah.txt open.

[JonathanM]

5 minutes ago, Maddeen said:

Because there can't be a slave-setting for a drive that doesnt even exist.

The slave setting is for the mapping, not the drive.

[Maddeen]

3 minutes ago, jonathanm said:

The slave setting is for the mapping, not the drive.

For heavens sake - that was my missing link A press on the EDIT button and there it is... 🙈

[trurl]

29 minutes ago, Maddeen said:

I thought that this mapping is just a "link" -.- and IF there is any unassigned disk it would be presented there. So I left this mapping. 

But in my case - there was no unassigned disk - so the error message is misleading or wrong.

Because now I need to set up this mapping every time again when I want to browse through a unassigned drive

Otherwise the plugin will remind me again and again ...

You need to set this up slave so it will have the correct setting when you have an unassigned drive. Otherwise Krusader will not be able to work with that unassigned drive. If you have the mapping, which is reasonable so it will be available when you have an unassigned drive, then you must make the mapping slave or it won't work when you plug in an unassigned drive because if it isn't slave it can only work with disks that are already plugged in when you boot up (technically when the docker service starts).

 

1 hour ago, Maddeen said:

user = root folders + cache | user0 = root folders only

I would word this differently. The user shares ARE the root folders on cache and array. Cache itself has root folders that are part of user shares. user is all root folders on cache and array, user0 is only the root folders on array.

 

User shares are how Unraid allows folders to span disks. Each root folder on cache and array is part of a user share named for that root folder.

[Squid]

49 minutes ago, Maddeen said:

send you a beer

Thank you.  Always appreciated