AppData Contents Deleted after Reboot

[codeigniter]

Hi Guys,

 

    I've had Unraid running for about a year now, previously it's been rebooted several times to install updates, move cables etc and I haven't had an issue.  A few days ago a notification popped up saying there was a new version of Unraid available, so I installed it and the server rebooted automatically.  When it booted back up I'd lost the user created items in the AppData folders, so the configuration files for various dockers. 

 

I can't remember how I originally set up the server as it was quite a while ago, but I've rebooted it plenty of times and never had this issue, so either I've unknowingly changed something or updating Unraid has.  Anyway it's going to take me ages to restore all the stuff I've lost so I'd like to try and safeguard against it in the future.

 

I was reading up on the issue last night and as I don't have a cache drive I assume that the contents were being stored in RAM, although they clearly weren't before.  I know I can turn caching off completely, but it would be nice to have a cache drive to speed things up when I'm transferring movies or large files across, so the plan is to throw a cache SSD in. 

 

I was reading the docs yesterday and in a section on cache drives there's a part that seems contradictory to me, so hopefully someone can clarify.  In a section titled 'warm spare' or something like that, It says I need a cache drive at least the same size as my parity which would be 2TB, then just below that it says the cache needs to be big enough to hold anything transferred until the mover kicks in.  In the latter case a 250Gb SSD would be fine for me.  Do I need a cache the same size as parity?  Or is that only if I for some reason decide to use my SSD as a parity if the parity fails?

 

My other issue is the mover, if I install a cache drive will the default mover configuration automatically back up my cache to the array?  Once everything is configured it doesn't get changed very often, so backing up once a day at 3:04AM is fine.  I just don't want to assume it will and pay the price later on, as it clearly isn't backed up to the array now/anymore.

[Squid]

Post your diagnostics

[codeigniter]

I'm not near the machine to post the diagnostics, but I know the cause of the issue anyway, something has changed so those files were stored in RAM cache.  I'm asking about moving forward with a dedicated cache drive and the mover. 

[JorgeB]

14 minutes ago, codeigniter said:

something has changed so those files were stored in RAM cache.

That will happen if you were using a path not in /mnt/user, /mnt/disk# or /mnt/pool_name

[codeigniter]

Yeah I realise that now after looking into it yesterday, odd that I haven't lost anything before though.  The AppData directory is set to prefer cache, but the subdirectories vary from being on the array to being disk1 cache, all of my dockers are set to /mnt/user, I didn't specify which subdirectories should be on cache and which shouldn't so seems to be something automatic, I don't know, I just want to stop it happening again.

 

Do you know if I need a cache drive the same size as parity as per the docs or just big enough to store data between move operations?

Does the mover automatically back up my cache or do I have to configure something?  So if my cache failed could I just replace it and Unraid restores it from the array automatically?

[trurl]

1 hour ago, codeigniter said:

cache drive the same size as parity as per the docs

Where did you see that "per the docs"?

 

I've never heard anyone suggest cache needed to be as large as parity.  Cache isn't even part of the parity array. 

[trurl]

Not sure you've really got to the bottom of what happened. Diagnostics might give some clues 

[trurl]

2 hours ago, codeigniter said:

lost the user created items in the AppData folders, so the configuration files for various dockers

Not clear whether you mean the docker templates, which are on flash and contain the settings from the unraid webui to create the container. Or you mean the actual appdata each container creates for itself for its working data. 

 

How did you do the upgrade? 

[codeigniter]

Quote

I was reading the docs yesterday and in a section on cache drives there's a part that seems contradictory to me, so hopefully someone can clarify.  In a section titled 'warm spare' or something like that, It says I need a cache drive at least the same size as my parity which would be 2TB, then just below that it says the cache needs to be big enough to hold anything transferred until the mover kicks in.  In the latter case a 250Gb SSD would be fine for me.  Do I need a cache the same size as parity?  Or is that only if I for some reason decide to use my SSD as a parity if the parity fails?

 

49 minutes ago, trurl said:

Where did you see that "per the docs"?

 

I've never heard anyone suggest cache needed to be as large as parity.  Cache isn't even part of the parity array. 

 

Quote

As a Warm Spare: you will want a drive that is the same size or larger than your parity disk. This drive can be any speed you choose, just remember that in the event of a parity rebuild (if your parity disk dies) or a rebuild from parity (if a data disk dies), the process will proceed at the speed of your slowest disk.

Amount of data

 

The final consideration in choosing a cache drive is to think about the amount of data you expect to pass through it. If you write ~10 GBs per day, then any drive 10 GB or larger will do (a 30 GB SSD may be a good fit in this case). If you write 100 GB in one day every few weeks, then you will want a cache drive that is larger than 100 GB. If you attempt a data transfer that is larger than the size of your cache drive, the transfer will fail.

 

https://wiki.unraid.net/Cache_disk#Purpose

 

Above is the section I'm looking for clarification on, I assume it is what I said in my original post and it's only if I wanted to use it as a parity in the event parity fails?  But I assumed my files were stored on the array and they weren't, so I don't want to risk assuming again.

[codeigniter]

54 minutes ago, trurl said:

Not clear whether you mean the docker templates, which are on flash and contain the settings from the unraid webui to create the container. Or you mean the actual appdata each container creates for itself for its working data. 

 

How did you do the upgrade? 

 

Not the templates they're fine, for instance Frigate, which has a path of something like 'mnt/user/AppData/Frigate' on the Docker template, but there's nothing in this directory to begin with I have to create the config yaml file for the cameras etc, on reboot the directory is emptied.

 

With Home Assistant it has a similar path 'mnt/user/AppData/HomeAssistant' but there are a bunch of files in there by default, it's only the files added or modified after the initial installation of the docker that disappear on reboot.

 

Quote

How did you do the upgrade? 

 

Just clicked the pop up notification that said there was a new Unraid version and followed the prompts.

Edited October 18, 2022 by codeigniter

[Kilrah]

1 minute ago, codeigniter said:

'mnt/user/AppData/HomeAssistant'

Note that things are case-sensitive, "appdata" and "AppData" are not the same. Default is "appdata". Could cause issues if you have both of them used.

[codeigniter]

1 hour ago, trurl said:

Not sure you've really got to the bottom of what happened. Diagnostics might give some clues 

 

I'll pull the diagnostics when I get home and post them.

[codeigniter]

6 minutes ago, Kilrah said:

Note that things are case-sensitive, "appdata" and "AppData" are not the same. Default is "appdata". Could cause issues if you have both of them used.

 

Not sure whether I set up the AppData folder or if it was there already as it was soo long ago, but I only have 1 share 'AppData' and all the docker templates point to that in camel case.

 

Sorry just tried to VPN onto server to pull diagnostics and I'm wrong it's 'appdata' lower case and everything points to that in lower case, apologies.

Edited October 18, 2022 by codeigniter

[codeigniter]

1 hour ago, trurl said:

Not sure you've really got to the bottom of what happened. Diagnostics might give some clues 

 

Diagnostics attached.

tower-diagnostics-20221018-1513.zip

[wgstarks]

16 minutes ago, codeigniter said:

Above is the section I'm looking for clarification on, I assume it is what I said in my original post and it's only if I wanted to use it as a parity in the event parity fails?  But I assumed my files were stored on the array and they weren't, so I don't want to risk assuming again.

That section would only apply if you wanted to use the drive as a possible replacement for parity. I think speed is more of a consideration for most people rather than size. SSDs are common for this. My parity is 16TB so I’m not going to use a cache drive anywhere near that size.😁

[codeigniter]

2 minutes ago, wgstarks said:

That section would only apply if you wanted to use the drive as a possible replacement for parity. I think speed is more of a consideration for most people rather than size. SSDs are common for this. My parity is 16TB so I’m not going to use a cache drive anywhere near that size.😁

 

My thoughts exactly, thanks for confirming, if the cache needed to be the same size as parity for some reason, I'd have just turned caching off...

[trurl]

2 hours ago, codeigniter said:

the section I'm looking for clarification on

Maybe this section at the very top of that page will help:

Quote

Important! This page is a basic introduction to the unRAID Cache drive, but was written for v4 and v5. There is no mention of Dockers, VM's, or Cache Pools. For v6 users, this page serves as an introduction, but you should also check these resources

The section you were reading was probably written before SSDs were common and HDDs were 2TB or less.

 

I doubt anybody thinks about "warm spare" these days since SSDs are so much better than HDDs for cache.

[trurl]

2 hours ago, Kilrah said:

Note that things are case-sensitive

Not possible to tell for sure with anonymized diagnostics but it looks like you may have a Media share and a media share. SMB isn't case sensitive and will only show one of these on the network.

 

 

[trurl]

12 minutes ago, trurl said:

Dockers, VM's, or Cache Pools

https://wiki.unraid.net/Manual/Shares#Default_Shares

 

Since you enabled Docker without cache, the default shares have been created on the array. When you install a fast pool (cache) you will have to disable Docker in Settings to get these moved to fast pool.

[trurl]

5 hours ago, codeigniter said:

AppData directory is set to prefer cache, but the subdirectories vary from being on the array to being disk1 cache, all of my dockers are set to /mnt/user, I didn't specify which subdirectories should be on cache and which shouldn't so seems to be something automatic, I don't know, I just want to stop it happening again.

Can you clarify this since you don't actually have a cache drive?

[codeigniter]

2 hours ago, trurl said:

Maybe this section at the very top of that page will help:

The section you were reading was probably written before SSDs were common and HDDs were 2TB or less.

 

I doubt anybody thinks about "warm spare" these days since SSDs are so much better than HDDs for cache.

 

Yeah I assumed that would be the case, but I wanted to get a NAS grade NVME drive and I'm not prepared to pay £300-£400 for a 2TB one, £50-£60 for a 250GB is fine, I've ordered a 250GB now.

[codeigniter]

2 hours ago, trurl said:

Can you clarify this since you don't actually have a cache drive?

 

 

Apologies I thought the anonymised diagnostics would just strip IP's and MAC addresses etc.

 

When I initially set up my server I only wanted a box to run docker containers on, there were other options, but for various reasons I elected not to go with any of those.  So I confess I only read up on configuring Unraid to get it running to the extent that I needed it to.  Up until now it hasn't let me down and there's been no need for me to dig any deeper.

 

I only have 3 drives in the box, 1 parity, 1 data drive this is my array, and a third surveillance class drive that is unassigned I use in Frigate for recordings etc.  Definitely no cache drive, caching wasn't something I concerned myself with as I don't have a drive for it, so I just assumed nothing was cached.  

 

When I say Disk 1 cache I'm going off what it says in the location column when I navigate the shares directory tree, right now there are only a couple of files showing Disk 1 cache, which I assume are temporary files in RAM or something, but before everything was deleted there were a few files with that location.  I'm guessing that my lost files were Disk 1 cache, I haven't recreated any of the lost data yet, I'm gonna wait until my cache drive comes tomorrow.

[Kilrah]

There is no cache unless you created a pool with drives... 

 

What does your main tab look like? Post a screenshot

[codeigniter]

2 hours ago, trurl said:

https://wiki.unraid.net/Manual/Shares#Default_Shares

 

Since you enabled Docker without cache, the default shares have been created on the array. When you install a fast pool (cache) you will have to disable Docker in Settings to get these moved to fast pool.

 

So going off what you're saying here I shouldn't have lost any data as it should have been on the array.  Does the flash drive store anything related appdata, like pointers or anything, could it be the flash drive failing?

[codeigniter]

9 minutes ago, Kilrah said:

There is no cache unless you created a pool with drives... 

 

What does your main tab look like? Post a screenshot

 

Screenshot