-C-

Just installed the Intel GVT-g plugin, hoping for better VM performance. The first line of the description is "Intel-GVT-g is a technology that provides mediated device passthrough for Intel iGPUs (Broadwell and newer).", then a load of benefits. I had no idea what generation a Broadwell CPU is, so had to look it up- it's 5th gen, I have 12th gen. Sounds good, let's go. So I installed it and it got me to reboot after checking the existence of a line in a file on the boot drive. All good, rebooted, checked the plugin's settings again and there's this message: GVT-g not supported GVT-g is currently only supported on: Client platforms: 5th, 6th, 7th, 8th, 9th and 10th Generation Intel® Core Processor Graphics Server platforms: E3-v4, E3_v5, E3_v6, E-2xxx, W-12xx, W10xxx Intel® Processor Graphics". It would be really good if this information was included in the App description to avoid us wasting our time. Also, is there an equivalent for > 10th gen CPUs? Thanks

Nice- is there any chance you'll be updating for v7?

Heya- Long shot, but any chance you signed up for Unraid Connect? You log in via https://account.unraid.net If so, you might find a backup of your USB in there.

Thanks OP- have just successfully followed your guide to update my 990 Pro With Heatsink. Can confirm that I needed to reboot before the Unraid UI was showing the updated firmware version. Hoping the new firmware will stop the drive from randomly crashing & disappearing from Unraid.

Marking this as solved- Not had the issue recur, but then again- this only happened the once and months after Immich had been set up. Will report back if it does.

Thanks, I have 32GB of RAM, so surprised that this is an issue considering that Immich recommend 6GB. I have followed the instructions here and limited Immich to have 16GB of RAM and will see how this affects things.

First time I've seen this in the nearly 2 years that I've been running Unraid, was picked up by the Fix Common Problems and following the advice there I'm posting my diagnostics here for help with this. tower-diagnostics-20240724-1149.zip Thanks

In case it's relevant/ helpful, I've been having similar troubles. Fitted a 4tb 990 Pro with factory heatsink into M2_1 (hadn't used that slot before as I was saving it for a fast drive). Same deal with my motherboard, although it's Intel; M2_1 is tied directly to the Raptor Lake 1700 socket. The other 3 run via the chipset. Moved my appdata and Immich libraries onto it to get the best performance I could. It ran fine for more than a month, then about 3 weeks ago I had the system lock up. Had to hard-reset to get it back. Then it happened again a week or so later. Figured out that the 990 had disappeared, checked the BIOS after rebooting and it wasn't there. But if I powered down and left it for a bit, it would reappear and work fine. Did so for a week, then during the night last night it happened again. Been searching around today and found a fair few posts from folk having similar issues. In this one, they said they resolved the issue by switching the 2 NVMes. So today I've moved mine from M2_1 to M2_4 (I already had a 1tb WD Black in M.2_2 & 3, which have been fine for the 18 months since I built the server). I'm not sure if mine's dying due to high load, but probably. Most of my server's heavy lifting (backups, snapshots, mover) are running in the night, and each time the drive has died it's been overnight. Looking like this could be an issue with 990 NVMes.

Have now reformatted the slower drive as ZFS to accept the snapshots. Glad to see that this hasn't affected its speed, although I will be keeping an eye on it. For the first time in a very long time, when I checked Unraid first thing yesterday (a couple of days after setting up the new snapshotting, after things had settled), all the drives were spun down. Since then though, every time I check the drive access, I'm seeing constant disk reading and writing affecting all drives, so I need to investigate this further. It's possible that it's just related to me accessing the GUI...

Yes, certainly strange. I followed Space Invader One's guide to use his ZFS Snapshot and Replication Script for snapshotting from cache to the ZFS disk array. Although that all seemed to work OK, It's possible that I messed something up during setup. It's also possible it could be something else to do with the various ZFS systems required for the script to function. Maybe FUSE didn't like something? I really don't know, but I doubt it was to do with the filesystem, more likely an issue/ glitch in something that was interacting with it.

Thanks Jorge- I've reformatted the drive as XFS and run a new speedtest: The 10/7/23 & 5/5/24 lines are pretty much exactly on top of each other. So my hunch turned out to be correct- it was a software issue, which is good as I don't have to worry about returning/ replacing. It's the fastest spinning drive in the server too, so good to have it back to its former sprightliness . My next step is to move the data from the 10TB drive onto this one, then reformat it to ZFS and set it up to receive snapshots from the ZFS cache drives. I've just run a DiskSpeed test on it, which gave the same result as all previous tests. Will be interesting to see if I have any issues with it once it's formatted as ZFS and receiving snapshots. Edit: To say that the server generally is feeling much more responsive again. There was definitely a system-wide issue related to the fact that this drive was constantly being accessed, crushing the system's performance. The Load Average was always 4 or more, even at idle. now it's back down to under 1!

It took over a week, but it's finished without error: SMART Extended Self-test Log Version: 1 (1 sectors) Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed without error 00% 7154 This is the only ZFS formatted drive in the array, formatted as ZFS so I could take snapshots of ZFS cache drives. During the extended SMART test I could see the disk IO was pretty much constant using htop. Only on this disk, and as said previously, this drive only has backups, archives and snapshots; nothing that should be being accessed, so I don't see why there's constant drive IO. It's the only drive that behaves like this. I think what I'd like to do is swap things around- I have a slow 10TB drive that's currently in the array being used as a media drive I'd like to reformat in ZFS and use for backups, archiving and snapshots. I'd like to format this 18TB drive as XFS and put media shares on it. Makes most sense to me if I reformat the 18TB drive as XFS, move the media from the 10TB drive onto it, then reformat the 10TB drive as ZFS. Can you suggest the easiest/ best way to achieve this? Thanks

OK, thanks. I will remove it and see how that goes. I don't have a spare drive and prices are not favourable at the moment, so I won't be buying another. If this drive's bad, I'd like to get it replaced by Amazon or the manufacturer. My worry is- how do I prove there's an issue when there are no SMART errors or anything other than poor performance to show there's an issue with it?

Thanks Jorge, I don't know much about the inner workings, but I figure if I can take this disk out of use and try deleting all links to it from Unraid, then try again as if it was a fresh disk to Unraid- if it continues to have issues then I know it's disk issues, if not then it was a software issue. It certainly isn't happy as it is and causing system-wide issues, so I have to do something. Haha- oh yes, it was late! Still shouldn't have been anything accessing that drive (the only things that are left on there are old archive files), yet it's had this constant read rate at idle for a while now. Nothing listed for this HDD under disk activity or Open Files. Can you help with how to remove, clear and replace/ retry? Thanks

Following multiple strange issues with my server and increasing weird things going on with this drive, I became ever more suspicious of it. Some examples: Unreliable reading or writing and huge usage of resources when attempting to unable to copy files off it using Unbalanced (it would eventually time out while doing its initial scan of certain directories and return to its start page). Strange permissions issues or permissions changing and trying to set a chmod command would hang indefinitely. If I tried to get the size of a directory using File Manager's calculate button on certain directories, it would hang indefinitely. There are some directories I've successfully used Unbalanced to clear files out of and onto another drive, but now can't delete the empty directories, even via CLI. So I've been moving data off. Long, slow process as I'm not sure whether a move is going slowly due to large backup image files, or whether it has hung. I've nearly finished and what's left on there is of low importance, so I'm about to remove the drive. My suspicion though is that there's something wrong with Unraid's management of the file system/ FUSE regarding this particular disk and it's been causing huge performance issues system-wide. For another clue to something being awry, have a look at the transfer rate of it, taken a few minutes ago (there was no actual activity on that drive that I know of): This is a regular SATA HDD- there's no way it could ever attain that kind of speed. Now that I've got any valuable data off and am at a point of being able to clear it, what is the best method of clearing the drive completely so that any FUSE linking to it is destroyed and I can try it again as if it was a fresh drive to Unraid?

I followed this YouTube video to install Immich- Used the docker apps from the standard Unraid Apps page: Immich Postgres 15 Redis I configured them as directed, only a minimal amount of config was needed in the Docket templates, but Immich wouldn't start up as it was meant to. I didn't have much time to troubleshoot, so gave up for the day. I found the following day that Nextcloud wouldn't start, and had numerous issues since: Nextcloud wasn't available. I found that MariaDB-Official wouldn't start. The solution to getting that running again seemed to be a combination of reverting to an earlier ZFS snapshot, installing the rc version, then multiple directories in the docker were read-only: 2024-02-27 20:37:36 0 [Warning] Can't create test file '/var/lib/mysql/1dd699d8d500.lower-test' (Errcode: 13 "Permission denied") 2024-02-27 20:37:36 0 [Note] Starting MariaDB 11.4.1-MariaDB-1:11.4.1+maria~ubu2204-log source revision fa69b085b10f19a3a8b6e7adab27c104924333ae as process 28 2024-02-27 20:37:36 0 [ERROR] mariadbd: File './binlog.index' not found (Errcode: 13 "Permission denied") 2024-02-27 20:37:36 0 [ERROR] Aborting 2024-02-27 20:38:09+00:00 [ERROR] [Entrypoint]: Unable to start server. 2024-02-27 20:37:27+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.4.1+maria~ubu2204 started. 2024-02-27 20:37:35+00:00 [Note] [Entrypoint]: MariaDB upgrade information missing, assuming required 2024-02-27 20:37:35+00:00 [Note] [Entrypoint]: Starting temporary server 2024-02-27 20:37:35+00:00 [Note] [Entrypoint]: Waiting for server startup Once I'd chmod 777'd the directories, it started and was OK. Once that was fixed, I was getting this error when trying to load Nextcloud: To fix that, I had to chmod 777 those directories. Following that, Nextcloud would start up, but it's running far slower than it used to. Plex was running fine initially, albeit loading pages slower than usual and the content on its home screen not displaying on first load, but would load after going to another page and then back again. Then a few days ago, it stopped playing files altogether. I got a "Critical: libusb_init failed" in its log. I managed to fix this by doing a "force update" from the Docker page. It pulled some files and worked fine for a few hours. Now today when we tried it again, it's loading pages, but not playing any files, the "Critical: libusb_init failed" error is back, and doing a "force update" isn't pulling files or making any difference. Generally, everything feels slower. I have tried rebooting twice while trying to troubleshoot since this began, each time I got an unclean shutdown. I understand that it could be complete coincidence that things went wrong just around the time that I'd tried to install Immich, and I'm not saying 100% that's what caused any of this, but as far as I'm aware, that's the only thing that I'd touched on the server before these issues started happening. Weirdnesses: How did installing 3 Docker apps affect anything else? Isn't the whole point of Docker that it's containerised and messing with an App shouldn't be able to affect anything else? I particularly don't get how the directory permissions in appdata of unrelated Docker apps (MariaDB and Nextcloud) were changed when I'd never had any need to go into those directories. Even if it wasn't anything to do with installing Immich, how could those permissions change? After I restored from the Mariadb ZFS snapshots, I still had permissions issues. I didn't think this was possible- shouldn't a snapshot restoration put everything back to exactly how it was when the snapshot was taken? How was Plex slow, but usable for a few weeks, then required a "force reinstall" to start working, which only worked temporarily and now it won't play media at all? tower-diagnostics-20240312-0057.zip I also have syslog enabled, so have logs of the whole timespan.

Thanks- I'm not seeing the password on restart, but I was able to log in with the nextcloud acct rather than root. I'll figure out the root password another time.

Unfortunately, that hasn't worked- rolled back to an earlier snapshot, then did as you said but didn't get a new version when I switched to :latest Everything looks to start up fine in the Mariadb log, but still not able to log in (tried console & Adminer).

Thanks- All files under /mnt/user/appdata/mariadb-official are now like this: -rwxrwxrwx 1 999 users 8192 Jan 23 2023 column_stats.MAD* Mariadb's running, but Nextcloud still isn't working. I'm getting this in the Mariadb log: 2024-02-28 22:49:12 14 [Warning] Aborted connection 14 to db: 'nextcloud' user: 'nextcloud' host: '172.17.0.1' (Got an error reading communication packets) I then tried logging in with Adminer, which got this error in Adminer and the same in the Mariadb log: Access denied for user 'root'@'172.17.0.1' (using password: YES) My understanding was that a ZFS snapshot would put everything back exactly as it was when the snapshot was taken. How could the permissions be different, and why would they have changed anyway when I haven't been anywhere near that directory? Any ideas?

I've now tried upgrading to 11.4.1-rc and downgrading to 11.2.3 and it's still refusing to start. Seems to be falling over on startup with the same "Can't create test file" error. Here's the complete log: 2024-02-27 20:37:36 0 [Warning] Can't create test file '/var/lib/mysql/1dd699d8d500.lower-test' (Errcode: 13 "Permission denied") 2024-02-27 20:37:36 0 [Note] Starting MariaDB 11.4.1-MariaDB-1:11.4.1+maria~ubu2204-log source revision fa69b085b10f19a3a8b6e7adab27c104924333ae as process 28 2024-02-27 20:37:36 0 [ERROR] mariadbd: File './binlog.index' not found (Errcode: 13 "Permission denied") 2024-02-27 20:37:36 0 [ERROR] Aborting 2024-02-27 20:38:09+00:00 [ERROR] [Entrypoint]: Unable to start server. 2024-02-27 20:37:27+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.4.1+maria~ubu2204 started. 2024-02-27 20:37:35+00:00 [Note] [Entrypoint]: MariaDB upgrade information missing, assuming required 2024-02-27 20:37:35+00:00 [Note] [Entrypoint]: Starting temporary server 2024-02-27 20:37:35+00:00 [Note] [Entrypoint]: Waiting for server startup I've got multiple daily appdata backups going back months- from both the appdata backup plugin, and ZFS snapshots. I've tried reverting to a snapshot from before the problem, but it's not making any difference. I've never had to repair/ restore a broken docker app before. Is there something I'm missing? Could the snapshot restoration not be working because I've changed the versions in the docker script which is causing a mismatch? Would really appreciate some help with getting this back up as I have a few other things (mostly Nextcloud) that are depending on it. Thanks

Could be a dodgy drive. Have a read of this:

I've checked for updates on the Docker page, it's saying I'm on the latest version, but when I check the log I seem to be stuck on 11.3.2: 2024-02-25 15:34:48 0 [Warning] Can't create test file '/var/lib/mysql/37ddd26137c4.lower-test' (Errcode: 13 "Permission denied") 2024-02-25 15:34:48 0 [Note] Starting MariaDB 11.3.2-MariaDB-1:11.3.2+maria~ubu2204-log source revision 068a6819eb63bcb01fdfa037c9bf3bf63c33ee42 as process 1 How do I get 11.4.0? Thanks Edit: I've tried added the post arguments with 11.3.2and still getting the errors.

I think this is because it's now outside the share/fuse appdata location. I'd try changing the appdata backup Appdata source(s) to use mnt/cache rather than mnt/user/appdata or whatever share location you're using for appdata.

For anyone else having this problem, I've finally managed to fix it. It turned out to be due to the Groupfolders app. I was alterted to this from this error; Some files have not passed the integrity check. Further information on how to resolve this issue can be found in the documentation ↗. (List of invalid files… / Rescan…) When I click the List of invalid files link, I get to <<mynextcloud.url>>/settings/integrity/failed And that's showing: Technical information ===================== The following list covers which files have failed the integrity check. Please read the previous linked documentation to learn more about the errors and how to fix them. Results ======= - groupfolders - FILE_MISSING - l10n/hy.json - l10n/ia.js Followed by a long list of files. So I could see there was an issue with the Groupfolders app. I updated it through the Nextcloud interface to the latest version and it didn't resolve the issue. But when I disabled the app, suddenly the clients on my devices started syncing and the WebDAV /CalDAV etc. errors were gone.

Thanks- I had these 3 that were well out of date: ┌────────────┬────────────┬────────────────────────────────────────────────────────────────────────┐ │ old date │ new date │ path │ ├────────────┼────────────┼────────────────────────────────────────────────────────────────────────┤ │ 2022-08-16 │ 2023-04-13 │ /config/nginx/nginx.conf │ │ 2022-08-20 │ 2024-01-03 │ /config/nginx/site-confs/default.conf │ │ 2022-08-20 │ 2023-08-13 │ /config/nginx/ssl.conf │ └────────────┴────────────┴────────────────────────────────────────────────────────────────────────┘ Have now replaced all 3 with the latest samples and that warning's not displayed in the startup log, but unfortunately it hasn't resolved the WebDAV issue.