Hi,

 

For the past few days I've had issues with some VM's pausing - eg I'd wake up, and overnight the VM I use for Pihole is paused. Starts with no trouble.

 

More recently, (past couple of days) I've had all my docker containers stop themselves. I restarted the ones I have as auto-start by disabling and re-enabling docker.

 

I've tried to look at some logs, but I think I might be looking in the wrong place.

 

My cache drive was kinda full, but I cleared that back to under 70%, and it hasn't resolved the issue.

 

Any ideas where I can start troubleshooting this?

 

Edit: I'm on 6.5.3, and I've restarted the server.

Edited September 12, 20187 yr by revilo951

Every time I've seen VMs pause is because there's not enough space, next time it happens grab and post the diagnostics: Tools -> Diagnostics

Looks like it might be my cache drive filling up. How can I stop that?

tower-diagnostics-20180912-2124.zip

Just now, revilo951 said:

Looks like it might be my cache drive filling up.

Yep, completely filled up.

 

1 minute ago, revilo951 said:

How can I stop that?

Get a bigger one? Or move some that from to the array.

 

Don't have a bigger one. What's the best way to move data to the array?

24 minutes ago, revilo951 said:

Don't have a bigger one. What's the best way to move data to the array?

The normal way is to run mover and that will move any files that belong to a User Share with a setting of Use Cache=Yes from the cache to the array.

 

it is also good practise to set a value for Min Free Space under Settings->Global Share Settings that is larger than the biggest file you want in the User Shares.    When free space on the cache drops below this value new files will be written directly to the array (albeit probably with a performance penalty) bypassing the cache.

What itimpi said.

 

I would just add that if you are constantly overfilling cache you are going to break things. Looks like your cache drive is plenty large. Maybe you should reconsider how you are using cache.

 

A couple of your user share settings are a bit unusual. Are you sure you know what they mean?

 

appdata:

• shareInclude="disk1"
• shareUseCache="prefer"

Those settings for appdata might make sense. I guess the way unRAID should interpret that is to try to keep appdata on cache but restrict any temporary overflow to disk1. Not sure that has ever been tested though since I have never noticed anyone using settings like that.

 

 

p-------e (anonymized name):

• shareExclude="disk1,disk2"
• shareUseCache="only"

A cache-only share excludes ALL array disks regardless.

Might be worth your trouble to study this link:

 

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?page=2#comment-537383

 

That's interesting.

 

My setup is non-typical - I have a cache SSD, and my other two disks are actually RAID arrays - Disk 1 is a RAID 1 of 2 1TB 10K RPM SAS drives, and Disk 2 is a RAID 6 of 6 8TB SATA drives.

 

My hardware doesn't allow individual disk passthrough ¯\_(ツ)_/¯

 

How would you set up shares in my situation?

3 minutes ago, revilo951 said:

That's interesting.

 

My setup is non-typical - I have a cache SSD, and my other two disks are actually RAID arrays - Disk 1 is a RAID 1 of 2 1TB 10K RPM SAS drives, and Disk 2 is a RAID 6 of 6 8TB SATA drives.

 

My hardware doesn't allow individual disk passthrough ¯\_(ツ)_/¯

 

How would you set up shares in my situation?

Ideally you would get different hardware so unRAID could work with the individual disks as intended.

 

But taking what you have, those details about some of your disks actually being RAIDs themselves seems irrelevant to the question of how to configure your shares to use cache.

 

Cache is intended as a way to improve performance. That could mean read/write performance of dockers and VMs, or it could mean write performance of data that would later be moved to the array. Or it could even mean read/write performance of data that stays there.

 

Different people make different tradeoffs for their purposes.

 

For example, I use cache for dockers (no VMs on my system) and for a copy of a subset of my pictures and music to allow other devices on my LAN to read them without spinning any drives.

 

I also have a cache-prefer DVR share for performance reasons. Any recording kept long term I move to the array manually.

 

I don't cache any of my other user shares. They are mostly backups and downloads, so they are written by unattended processes and I don't care if the writes are a little slower. And they are immediately protected by parity since they are never on cache.

 

No doubt your uses will be somewhat different.

 

Did you study that link I gave?

Do you have (an unRAID) parity drive? If not then there is, of course, no write performance penalty with writing to the array, though writing to SSD might be faster than writing to spinners (or not since you are striping). Unless you have 10G ethernet you are network-bound anyway.

No parity drive. however, I have a DAS that I'm setting up. It has 12 x 3TB drives. I'll set two of them for parity. It'll be interesting to see how unRAID handles 1 x 1TB drive, 1 x 30TB drive and 12 x 3TB drives.

 

Any suggestions for when I do that? Right now, changing the RAID drives to JBOD isn't an option, unfortunately.

7 hours ago, revilo951 said:

No parity drive. however, I have a DAS that I'm setting up. It has 12 x 3TB drives. I'll set two of them for parity. It'll be interesting to see how unRAID handles 1 x 1TB drive, 1 x 30TB drive and 12 x 3TB drives.

 

Any suggestions for when I do that? Right now, changing the RAID drives to JBOD isn't an option, unfortunately.

You are aware that since you are presenting a 30TB volume to unraid as a single disk, you will need a second 30TB volume to be able to use unraid's parity protection, right? With the drives you have outlined, the only logical (to me) layout is using the 12 drives in the parity protected array, and keeping your two current RAID volumes as unassigned devices. You will lose write speed to your RAID1 if it's part of the unraid parity scheme, and you will lose the entire capacity of the RAID6, because it will have to be one of the parity drives. You can't have data slots with larger capacity than either of the parity slots.

20 minutes ago, jonathanm said:

With the drives you have outlined, the only logical (to me) layout is using the 12 drives in the parity protected array, and keeping your two current RAID volumes as unassigned devices.

I think jonathanm has a good plan for you here.

That makes sense

 

Thanks folk!

16 hours ago, revilo951 said:

my other two disks are actually RAID arrays - Disk 1 is a RAID 1 of 2 1TB 10K RPM SAS drives, and Disk 2 is a RAID 6 of 6 8TB SATA drives.

 

My hardware doesn't allow individual disk passthrough ¯\_(ツ)_/¯

On further thought, I can't in good conscience leave this as is.

 

Unraid does not have an easy method of keeping up with the health of RAID arrays presented in bulk by hardware RAID cards, it is designed to manage each disk individually.  This means that it is on you to keep up with the status of the drives in those pools, and when you have a drive failure, unraid won't be able to warn you. When you exceed the number of allowed drive failures in that array, you will lose ALL the data on it, and unraid will not have been able to warn or protect you.

 

What others in your situation have done is create individual RAID0 arrays for each of your disks, then allow unraid to use those individual disks relatively normally. This at least allows unraid to handle complete drive failures gracefully if they are part of the parity protected array. SMART status and predictive health isn't automatically supported, but at least when a drive fails, it can be properly emulated.

 

The downside is that your drive speeds won't benefit from striping, you will be limited to the read throughput of each drive individually, and write performance will be roughly halved.

 

Also, my suggestion of leaving your RAID arrays as unassigned devices has its drawbacks as well. Unassigned devices don't participate in user shares. You have to set up mount points and keep up with them yourself.

 

You are stepping outside the normal usage for unraid, and thus the general advice that you find here on the forums may not apply to your setup. Some things that are easy to do if you follow the norms will be either not possible or difficult to accomplish on your RAID volumes.

 

Unraid and RAID only hardware do NOT mix well.

 

You have been warned. RAID and unraid are NOT a replacement for a good backup plan. Data loss has many causes, but your configuration has more risk than most.

 

If you have a high tolerance for learning new things, going out on your own and forging new paths, then by all means, continue as planned. If you would rather take the easy path and follow where many have already been, then you need to source hardware that is tried and tested to work well.

2 hours ago, jonathanm said:

On further thought, I can't in good conscience leave this as is.

+1

Thankyou for the warning. I'm aware of how hardware RAID works, and get warnings from the iDRAC about the disk status'. Anything critical is replicated externally.

 

I'm also aware this configuration is highly unconventional. It has worked well for me until now.

 

Now that I have learnt more about how unRAID handles individual disks (didn't do a whole heap of reading when I first set it up due to the hardware RAID), I'm going to try and find a way to connect the backplane the has the existing RAID drives in JBOD, instead of the current setup.

 

FYI it's an R720 with a PERC H710 mini. The JBOD controller I have is an LSI 9201-16e flashed to IT mode. I guess I'd need an adaptor or something to go from the internal card connector type, to the external style.

 

Thanks again for your warning. Too many folk consider RAID to be backup.