codeigniter Posted October 18, 2022 Share Posted October 18, 2022 Hi Guys, I've had Unraid running for about a year now, previously it's been rebooted several times to install updates, move cables etc and I haven't had an issue. A few days ago a notification popped up saying there was a new version of Unraid available, so I installed it and the server rebooted automatically. When it booted back up I'd lost the user created items in the AppData folders, so the configuration files for various dockers. I can't remember how I originally set up the server as it was quite a while ago, but I've rebooted it plenty of times and never had this issue, so either I've unknowingly changed something or updating Unraid has. Anyway it's going to take me ages to restore all the stuff I've lost so I'd like to try and safeguard against it in the future. I was reading up on the issue last night and as I don't have a cache drive I assume that the contents were being stored in RAM, although they clearly weren't before. I know I can turn caching off completely, but it would be nice to have a cache drive to speed things up when I'm transferring movies or large files across, so the plan is to throw a cache SSD in. I was reading the docs yesterday and in a section on cache drives there's a part that seems contradictory to me, so hopefully someone can clarify. In a section titled 'warm spare' or something like that, It says I need a cache drive at least the same size as my parity which would be 2TB, then just below that it says the cache needs to be big enough to hold anything transferred until the mover kicks in. In the latter case a 250Gb SSD would be fine for me. Do I need a cache the same size as parity? Or is that only if I for some reason decide to use my SSD as a parity if the parity fails? My other issue is the mover, if I install a cache drive will the default mover configuration automatically back up my cache to the array? Once everything is configured it doesn't get changed very often, so backing up once a day at 3:04AM is fine. I just don't want to assume it will and pay the price later on, as it clearly isn't backed up to the array now/anymore. Quote Link to comment
Squid Posted October 18, 2022 Share Posted October 18, 2022 Post your diagnostics Quote Link to comment
codeigniter Posted October 18, 2022 Author Share Posted October 18, 2022 I'm not near the machine to post the diagnostics, but I know the cause of the issue anyway, something has changed so those files were stored in RAM cache. I'm asking about moving forward with a dedicated cache drive and the mover. Quote Link to comment
JorgeB Posted October 18, 2022 Share Posted October 18, 2022 14 minutes ago, codeigniter said: something has changed so those files were stored in RAM cache. That will happen if you were using a path not in /mnt/user, /mnt/disk# or /mnt/pool_name Quote Link to comment
codeigniter Posted October 18, 2022 Author Share Posted October 18, 2022 Yeah I realise that now after looking into it yesterday, odd that I haven't lost anything before though. The AppData directory is set to prefer cache, but the subdirectories vary from being on the array to being disk1 cache, all of my dockers are set to /mnt/user, I didn't specify which subdirectories should be on cache and which shouldn't so seems to be something automatic, I don't know, I just want to stop it happening again. Do you know if I need a cache drive the same size as parity as per the docs or just big enough to store data between move operations? Does the mover automatically back up my cache or do I have to configure something? So if my cache failed could I just replace it and Unraid restores it from the array automatically? Quote Link to comment
trurl Posted October 18, 2022 Share Posted October 18, 2022 1 hour ago, codeigniter said: cache drive the same size as parity as per the docs Where did you see that "per the docs"? I've never heard anyone suggest cache needed to be as large as parity. Cache isn't even part of the parity array. Quote Link to comment
trurl Posted October 18, 2022 Share Posted October 18, 2022 Not sure you've really got to the bottom of what happened. Diagnostics might give some clues Quote Link to comment
trurl Posted October 18, 2022 Share Posted October 18, 2022 2 hours ago, codeigniter said: lost the user created items in the AppData folders, so the configuration files for various dockers Not clear whether you mean the docker templates, which are on flash and contain the settings from the unraid webui to create the container. Or you mean the actual appdata each container creates for itself for its working data. How did you do the upgrade? Quote Link to comment
codeigniter Posted October 18, 2022 Author Share Posted October 18, 2022 Quote I was reading the docs yesterday and in a section on cache drives there's a part that seems contradictory to me, so hopefully someone can clarify. In a section titled 'warm spare' or something like that, It says I need a cache drive at least the same size as my parity which would be 2TB, then just below that it says the cache needs to be big enough to hold anything transferred until the mover kicks in. In the latter case a 250Gb SSD would be fine for me. Do I need a cache the same size as parity? Or is that only if I for some reason decide to use my SSD as a parity if the parity fails? 49 minutes ago, trurl said: Where did you see that "per the docs"? I've never heard anyone suggest cache needed to be as large as parity. Cache isn't even part of the parity array. Quote As a Warm Spare: you will want a drive that is the same size or larger than your parity disk. This drive can be any speed you choose, just remember that in the event of a parity rebuild (if your parity disk dies) or a rebuild from parity (if a data disk dies), the process will proceed at the speed of your slowest disk. Amount of data The final consideration in choosing a cache drive is to think about the amount of data you expect to pass through it. If you write ~10 GBs per day, then any drive 10 GB or larger will do (a 30 GB SSD may be a good fit in this case). If you write 100 GB in one day every few weeks, then you will want a cache drive that is larger than 100 GB. If you attempt a data transfer that is larger than the size of your cache drive, the transfer will fail. https://wiki.unraid.net/Cache_disk#Purpose Above is the section I'm looking for clarification on, I assume it is what I said in my original post and it's only if I wanted to use it as a parity in the event parity fails? But I assumed my files were stored on the array and they weren't, so I don't want to risk assuming again. Quote Link to comment
codeigniter Posted October 18, 2022 Author Share Posted October 18, 2022 (edited) 54 minutes ago, trurl said: Not clear whether you mean the docker templates, which are on flash and contain the settings from the unraid webui to create the container. Or you mean the actual appdata each container creates for itself for its working data. How did you do the upgrade? Not the templates they're fine, for instance Frigate, which has a path of something like 'mnt/user/AppData/Frigate' on the Docker template, but there's nothing in this directory to begin with I have to create the config yaml file for the cameras etc, on reboot the directory is emptied. With Home Assistant it has a similar path 'mnt/user/AppData/HomeAssistant' but there are a bunch of files in there by default, it's only the files added or modified after the initial installation of the docker that disappear on reboot. Quote How did you do the upgrade? Just clicked the pop up notification that said there was a new Unraid version and followed the prompts. Edited October 18, 2022 by codeigniter Quote Link to comment
Kilrah Posted October 18, 2022 Share Posted October 18, 2022 1 minute ago, codeigniter said: 'mnt/user/AppData/HomeAssistant' Note that things are case-sensitive, "appdata" and "AppData" are not the same. Default is "appdata". Could cause issues if you have both of them used. Quote Link to comment
codeigniter Posted October 18, 2022 Author Share Posted October 18, 2022 1 hour ago, trurl said: Not sure you've really got to the bottom of what happened. Diagnostics might give some clues I'll pull the diagnostics when I get home and post them. Quote Link to comment
codeigniter Posted October 18, 2022 Author Share Posted October 18, 2022 (edited) 6 minutes ago, Kilrah said: Note that things are case-sensitive, "appdata" and "AppData" are not the same. Default is "appdata". Could cause issues if you have both of them used. Not sure whether I set up the AppData folder or if it was there already as it was soo long ago, but I only have 1 share 'AppData' and all the docker templates point to that in camel case. Sorry just tried to VPN onto server to pull diagnostics and I'm wrong it's 'appdata' lower case and everything points to that in lower case, apologies. Edited October 18, 2022 by codeigniter Quote Link to comment
codeigniter Posted October 18, 2022 Author Share Posted October 18, 2022 1 hour ago, trurl said: Not sure you've really got to the bottom of what happened. Diagnostics might give some clues Diagnostics attached. tower-diagnostics-20221018-1513.zip Quote Link to comment
wgstarks Posted October 18, 2022 Share Posted October 18, 2022 16 minutes ago, codeigniter said: Above is the section I'm looking for clarification on, I assume it is what I said in my original post and it's only if I wanted to use it as a parity in the event parity fails? But I assumed my files were stored on the array and they weren't, so I don't want to risk assuming again. That section would only apply if you wanted to use the drive as a possible replacement for parity. I think speed is more of a consideration for most people rather than size. SSDs are common for this. My parity is 16TB so I’m not going to use a cache drive anywhere near that size.😁 Quote Link to comment
codeigniter Posted October 18, 2022 Author Share Posted October 18, 2022 2 minutes ago, wgstarks said: That section would only apply if you wanted to use the drive as a possible replacement for parity. I think speed is more of a consideration for most people rather than size. SSDs are common for this. My parity is 16TB so I’m not going to use a cache drive anywhere near that size.😁 My thoughts exactly, thanks for confirming, if the cache needed to be the same size as parity for some reason, I'd have just turned caching off... Quote Link to comment
trurl Posted October 18, 2022 Share Posted October 18, 2022 2 hours ago, codeigniter said: the section I'm looking for clarification on Maybe this section at the very top of that page will help: Quote Important! This page is a basic introduction to the unRAID Cache drive, but was written for v4 and v5. There is no mention of Dockers, VM's, or Cache Pools. For v6 users, this page serves as an introduction, but you should also check these resources The section you were reading was probably written before SSDs were common and HDDs were 2TB or less. I doubt anybody thinks about "warm spare" these days since SSDs are so much better than HDDs for cache. Quote Link to comment
trurl Posted October 18, 2022 Share Posted October 18, 2022 2 hours ago, Kilrah said: Note that things are case-sensitive Not possible to tell for sure with anonymized diagnostics but it looks like you may have a Media share and a media share. SMB isn't case sensitive and will only show one of these on the network. Quote Link to comment
trurl Posted October 18, 2022 Share Posted October 18, 2022 12 minutes ago, trurl said: Dockers, VM's, or Cache Pools https://wiki.unraid.net/Manual/Shares#Default_Shares Since you enabled Docker without cache, the default shares have been created on the array. When you install a fast pool (cache) you will have to disable Docker in Settings to get these moved to fast pool. Quote Link to comment
trurl Posted October 18, 2022 Share Posted October 18, 2022 5 hours ago, codeigniter said: AppData directory is set to prefer cache, but the subdirectories vary from being on the array to being disk1 cache, all of my dockers are set to /mnt/user, I didn't specify which subdirectories should be on cache and which shouldn't so seems to be something automatic, I don't know, I just want to stop it happening again. Can you clarify this since you don't actually have a cache drive? Quote Link to comment
codeigniter Posted October 18, 2022 Author Share Posted October 18, 2022 2 hours ago, trurl said: Maybe this section at the very top of that page will help: The section you were reading was probably written before SSDs were common and HDDs were 2TB or less. I doubt anybody thinks about "warm spare" these days since SSDs are so much better than HDDs for cache. Yeah I assumed that would be the case, but I wanted to get a NAS grade NVME drive and I'm not prepared to pay £300-£400 for a 2TB one, £50-£60 for a 250GB is fine, I've ordered a 250GB now. Quote Link to comment
codeigniter Posted October 18, 2022 Author Share Posted October 18, 2022 2 hours ago, trurl said: Can you clarify this since you don't actually have a cache drive? Apologies I thought the anonymised diagnostics would just strip IP's and MAC addresses etc. When I initially set up my server I only wanted a box to run docker containers on, there were other options, but for various reasons I elected not to go with any of those. So I confess I only read up on configuring Unraid to get it running to the extent that I needed it to. Up until now it hasn't let me down and there's been no need for me to dig any deeper. I only have 3 drives in the box, 1 parity, 1 data drive this is my array, and a third surveillance class drive that is unassigned I use in Frigate for recordings etc. Definitely no cache drive, caching wasn't something I concerned myself with as I don't have a drive for it, so I just assumed nothing was cached. When I say Disk 1 cache I'm going off what it says in the location column when I navigate the shares directory tree, right now there are only a couple of files showing Disk 1 cache, which I assume are temporary files in RAM or something, but before everything was deleted there were a few files with that location. I'm guessing that my lost files were Disk 1 cache, I haven't recreated any of the lost data yet, I'm gonna wait until my cache drive comes tomorrow. Quote Link to comment
Kilrah Posted October 18, 2022 Share Posted October 18, 2022 There is no cache unless you created a pool with drives... What does your main tab look like? Post a screenshot Quote Link to comment
codeigniter Posted October 18, 2022 Author Share Posted October 18, 2022 2 hours ago, trurl said: https://wiki.unraid.net/Manual/Shares#Default_Shares Since you enabled Docker without cache, the default shares have been created on the array. When you install a fast pool (cache) you will have to disable Docker in Settings to get these moved to fast pool. So going off what you're saying here I shouldn't have lost any data as it should have been on the array. Does the flash drive store anything related appdata, like pointers or anything, could it be the flash drive failing? Quote Link to comment
codeigniter Posted October 18, 2022 Author Share Posted October 18, 2022 9 minutes ago, Kilrah said: There is no cache unless you created a pool with drives... What does your main tab look like? Post a screenshot Screenshot Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.