Stupot

Hi, I bought a couple of matching 8GB sticks of RAM to go from 16GB to 32GB on my server. I hadn't cleaned the system out in over a year so I took the opportunity to go at it with a hoover and dusting brush. After I was finished, it booted with disk1 in the array being disabled with a red x in the status icon. All the other drives appeared to be OK with green dots saying they were detected...initially! Then all of a sudden they all turned grey - which I assume meant they were not only "disabled", but they weren't being detected at all any more. I rebooted and tried again and the same thing with the disk1 drive being disabled and and others showing as OK...and it appeared they stayed OK. I started looking for how to resolve the disabled disk - and found a post saying to take the drive out the array, start in maintenance mode, stop array, then start array again and it should rebuild. So that is what I did...and at first I thought it was doing it - it said it was rebuilding the array...but all of a sudden the drives were all grey, and the number of errors on all the other drives except disk1 were going up like crazy with zero read/writes on any drive...yet it claimed it was rebuilding at 600-700mb/s? I've never had to do a rebuild so I wasn't sure what was happening but nothing seemed right so I stopped it. I then stopped the array and all the drives in the array now had red x's next to them and they were no longer selected from the drop down list. All the 4 of the affected drives are connected to an LSI HBA using a 4-port breakout cable. There are a couple other drives in there connected through the SATA ports on the motherboard and they seemed fine. I shutdown the system again and took the HBA and reseated it. I took it out when I cleaned the system so maybe it wasn't seated properly in the PCIE slot?! I also reseated all the RAM sticks as I thought maybe the 2 new sticks that I had put in hadn't seated properly (I also took out one of the originals just to check the label to be 100% sure they were matching sticks). The next thing I would have tried would be removing the 2 new sticks of RAM as perhaps they were faulty and causing errors with the HBA. I fired up the system again and thankfully, this time things seemed better. All the drives were showing up - disk1 had an orange exclamation next to it saying contents are being emulated...but it was better than the red X. I started the array again and...again, thankfully...the array started rebuilding disk1...and all the metrics are what I would expected - all the other drives are being read at 250mb/s and disk1 is being written to at the same speed - all with zero errors...so I think whatever it was is fixed? I think if I had shut down and reseated things before messing with removing the drive from the array and starting in maintenance mode, I would have avoided this? But if a 10 hour rebuild resolves things that I will take that as a win and consider it a lesson learnt. So, what do you guys think? Appreciate your time! And pray for me that the rebuild completes without issue and there are no more problems 😂

Hello, Just a quick one I think about something that happened this evening. Over the last couple weeks I've been doing upgrades and tinkering with my Unraid server. I've been shutting it down and rebooting it many times - and every time it has done so without issue. Tonight was different. I followed what I thought was my usual procedure when I'm shutting down: 1. Stop all docker containers (not the service though). 2. Stop my 1 running Linux VM (not the VM service though). 3. Press the shutdown buttons from Dashboard or "main". Once I get the "shutting down" web page, I close that page down and just wait for the server to shut off. I left it around 10 minutes tonight, and when I went into the room where the server is, it was still running. I went back onto the Unraid UI and everything was still accessible - array looked started etc - and I honestly thought that instead of confirming the shutdown command, I pressed "cancel" instead; so I just selected the shutdown button again - it asked if i wanted to shut down, and the server pretty much shut off immediately. When I booted it up again next time, everything was fine - I started the array - and then after a few minutes realised there was a parity check running. Because I assumed everything was hunky-dorry - I decided not to let the parity check continue - but then I noticed that in the few minutes it had been running - it actually found 17 sync errors - something that has never happened before after a parity check from unclean shutdown. I am obviously running another parity check right now (with the errors correcting) and it at 17% it has found another 9 errors. Oct 10 01:36:20 serverName kernel: md: recovery thread: P corrected, sector=2147483640 Oct 10 01:36:20 serverName kernel: md: recovery thread: P corrected, sector=2147483648 Oct 10 01:36:20 serverName kernel: md: recovery thread: P corrected, sector=2147483656 Oct 10 01:36:20 serverName kernel: md: recovery thread: P corrected, sector=2147483664 Oct 10 01:36:20 serverName kernel: md: recovery thread: P corrected, sector=2147483672 Oct 10 01:36:20 serverName kernel: md: recovery thread: P corrected, sector=2147483680 Oct 10 01:36:20 serverName kernel: md: recovery thread: P corrected, sector=2147483688 Oct 10 01:36:20 serverName kernel: md: recovery thread: P corrected, sector=2147483696 Oct 10 01:36:20 serverName kernel: md: recovery thread: P corrected, sector=2147483704 This was the last bit of log before it shutdown: Oct 9 22:24:47 serverName kernel: BTRFS info (device loop3): last unmount of filesystem 3f2f815a-d996-4e63-86a0-5c3b4d62cf4e Oct 9 22:24:47 serverName kernel: docker0: port 2(veth6baaa8b) entered disabled state Oct 9 22:24:47 serverName kernel: veth174b5d9: renamed from eth0 Oct 9 22:24:47 serverName kernel: docker0: port 2(veth6baaa8b) entered disabled state Oct 9 22:24:47 serverName kernel: device veth6baaa8b left promiscuous mode Oct 9 22:24:47 serverName kernel: docker0: port 2(veth6baaa8b) entered disabled state Oct 9 22:24:48 serverName root: stopping dockerd ... Oct 9 22:24:49 serverName root: ... Waiting to die. Oct 9 22:24:50 serverName root: ... Waiting to die. Oct 9 22:24:51 serverName root: ... Waiting to die. Oct 9 22:24:52 serverName root: ... Waiting to die. Oct 9 22:24:52 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:24:53 serverName root: ... Waiting to die. Oct 9 22:24:54 serverName root: ... Waiting to die. Oct 9 22:24:55 serverName emhttpd: shcmd (616435): umount --lazy /var/lib/docker Oct 9 22:24:58 serverName kernel: BTRFS info (device loop2): last unmount of filesystem c9cff771-8d35-4df3-8c1c-9171fbb3d9d8 Oct 9 22:24:58 serverName Recycle Bin: Stopping Recycle Bin Oct 9 22:24:58 serverName emhttpd: Stopping Recycle Bin... Oct 9 22:24:58 serverName unassigned.devices: Unmounting All Devices... Oct 9 22:25:03 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:25:14 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:25:25 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:25:37 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:25:48 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:25:59 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:26:10 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:26:21 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:26:32 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:26:43 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:26:54 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:27:06 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:27:17 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:27:28 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:27:39 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:27:50 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:28:01 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:28:12 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:28:24 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:28:27 serverName root: Status of all loop devices Oct 9 22:28:27 serverName root: /dev/loop1: [2049]:12 (/boot/bzfirmware) Oct 9 22:28:27 serverName root: /dev/loop0: [2049]:10 (/boot/bzmodules) Oct 9 22:28:27 serverName root: Active pids left on /mnt/* Oct 9 22:28:35 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:28:46 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:28:57 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:29:08 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:29:19 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:29:30 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:29:41 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:29:53 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:30:04 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:30:15 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:30:26 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:30:37 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:30:48 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:30:59 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:31:11 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:31:22 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:31:33 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:31:44 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:31:55 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:32:06 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:32:17 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:32:29 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:32:40 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:32:51 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:33:02 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:33:13 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:33:24 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:33:35 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:33:47 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:33:58 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:34:09 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:34:20 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:34:31 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:34:42 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:34:53 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:35:04 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:35:16 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:35:27 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name Oct 9 22:35:38 serverName key.dns_resolver: QNAP.LOCAL: No address associated with name "QNAP" is another NAS that I use UD to connect to - I shut that off before the Unraid server, but I have always done that. The only line that stood out to me was the "root: Active pids left on /mnt/*" - that could have been something hanging it? My disk settings shutdown time-out is 240s. Like I say, I have done this loads of times and it has never been an issue. I have a theory: I maybe thought more time had passed since my first telling the server to shutdown - and if I had just left it a little longer it would have shut down normally. Does Unraid force shutdown the server if you press the shutdown buttons in the GUI a second time? Anyway - re: the "sync errors". These are basically just bits on the parity drive that are not what they should be for what is on the array disks, right? It's not a case of any data loss/corruption or anything? Thanks. Apologies for the long-winded-ness 🙂

Thanks, everything makes sense now haha! Thank you for the tips too! I do use the file manager plugin for smaller transfers - but using rsync in the Krusader docker's terminal feels way more reliable for larger transfers. Now I've got the root access issue sorted hopefully will be my go to! Though will defo keep your "disown" trick in mind for future! Thank you!

Hey, I actually did the copying using a Krusader docker. When I first started using Unraid, I watched a lot of SpaceInvader One's guides - one of which was about moving data around. I needed to move data from my old NAS to the new one, and the method SIO explained was using Krusader, and it has worked fine whenever I needed to move data. Since it was so much data I was moving, I wanted a way of being able to see what is happening in the terminal window. As I understand it, if I had run the command in a shell window from Unraid GUI, I would have had to keep that window open for it to keep going. Using Krusader just seemed like a more stable way of doing it. I am making an educated guess here: In order for the Krusader docker to be able to copy the files while maintaining user and group, the docker needs root access, which is not enabled by default...and was not enabled when I copied the da;ta over. So I would guess that is why the files copied, but the user and group was not. Would likely also explain why it was nobody:users? When I ran "rsync -avhH" WITHOUT root access (as per the instruction from one of SIO's videoes) - everything copied with no errors. When I tried "rsync -avhogH" without access, there were loads of errors saying it didn't have permission to do something (I can't recall the exact wording) - trying the same command with root access enabled it worked fine and backups seem to be working spot on now. I'll be honest, I find it confusing understanding what a lot of the arguments for command line applications mean. I had no idea that "a" would essentially include o and g with it. Since it didn't error when copying the data, I assume that copying the user:group is an optional thing? Like "try and copy the user and group if you can but if you can't just skip it and give it a default user:group" type deal? Thanks for taking the time to read. One other thing: is there anything one can add to the end of your script that once it's finished it will change the owner of the .log files the script generates? They are owned by "root" as default and so am unable to open the log files over the network. Hopefully just a simple chown command for the log file(s) at the end of the script? Thanks!

Hello I have just bought a new 16TB drive to replace the 8TB drive I was using for incremental backups. I had hoped that I could just do a simple rsync copy of all the data on the original drive to the new drive (making sure to preserve all the hardlinks) and just make sure to change the share name of the drive to use in the script...and it would detect all the existing backups and continue from there. Unfortunately that doesn't appear to be happening. It's doing a full copy again, and from what I can tell from the drive metris, it's actually copying the data from itself? Like it appears to be copying the data from one of the backups on the drive and copying it to a new folder...not doing hard link? The drives where the source data is stored is not being hammered like one would expect if it were copying data from it. I am really confused. Any advice? Edit: I moved the data using rsync (rsync -avhH source dest). All the files seem to have the correct date stamps as the original drive location. The logs even show that it is picking up the last back up and using that as a basis...but is doing a full copy of the data instead of hard linking?! Could it have anything to do with the fact that the original 8TB drive I was using for your script was using BTRFS and with this new 16TB I decided to use XFS instead? EDIT AGAIN: I just checked the files again in the Unraid file manager - all the files are showing owned by "nobody"...whereas on the original 8TB drive, the files are owned by "stuart", my account name. Could that be cause? If so, is it solvable without having copy all the data again? Could I just use the chown command in some format to resolve this? ---------------------------------------------------------------------------------------- Edit again again: It's a file ownership issue. When I rysnc copied everything over, it has not kept the original files ownerships - so everything is now owned by "nobody". On the original 8TB drive, my files are a mix of "nobody" and "stuart" owned files. It would be impossible for me to go through and manually sort this. I am going to try and find a way to use rsync to transfer the ownerships to the data I copied so that I can avoid having to recopy it - it was 6.5TB of data so would be nice not to have to copy everything again 🥺 If anyone has any advice on how to accomplish this in case I don't succeed, that would be great. Thanks! ---------------------------------------------------------------------------------------- Edit again again again: Well, that was an adventure. Managed to sort it by adding +og to my rsync command. I just ran the exact same command as I did first time copying the data plus those additional arguments and it went through and copied the ownerships within 5 minutes. Incredibly impressed. rsync really is an impressive tool. I think my issues are resolved! last_backup: '20240929_060002/' Create incremental backup from /mnt/user/Documents to /mnt/disks/16TB_Backup_ZR712BR6/Documents/20241001_003439 by using last backup /mnt/disks/16TB_Backup_ZR712BR6/Documents/20240929_060002/ created directory /mnt/disks/16TB_Backup_ZR712BR6/Documents/.20241001_003439 cd..t...... ./

Hmm, I just tried ipleak.net and that yielded the same result - it showed the VPN IP...so I am not sure why you are having problems. Did you make sure you followed the above? For me it was all set up on the Docker side of things - I just switched to Wireguard, added the "--privileged=true" argument and then ran it and it connected first time. I then picked the PIA server I wanted and changed it in the wg0.conf file. That was it. There is some reference in that documentation about not all PIA endpoints supporting port forwarding. To be quite honest, I don't understand what an "endpoint" even is, beyond "it's a VPN server address"...and Googling "private internet access wireguard endpoints" didn't yield any results...such as a list...so I'm not sure how one is supposed to know which endpoints support port forwarding. Did you try a different server/endpoint address? Maybe the one you're connecting to is one of the ones that does not support port forwarding and that's why the IP leak test won't work? Not sure what implications that would be on actually trying to keep your IP hidden if it doesn't support port forwarding (again, my ignorance is that I don't fully appreciate what "port forwarding endpoint" even means lol).

Interesting. Feel like a bit of a schmuck now going all this time thinking I wasn't able to use all the bandwidth available to me 😂 When you say you're trying to run a leak test, what method(s) do you use? I've only been doing curl ifconfig.io in console and a Torguard check. I have never received any notices from my ISP slapping my wrists in the 10+ years of torrenting, so have always assumed everything has been golden.

I have just tried for the first time using Wireguard instead of OpenVPN as the client when using PIA. I have a 1gig fibre internet connection, but I have only ever been able to get 40mb/sec~ using PIA and OpenVPN. CPU was never maxed out so it always seemed like a PIA thing - I thought they were throttling me in some way. Using Wireguard, I am almost maxing out my connection when downloading, at around 90mb/sec. Does this sound right? Doing "curl ifconfig.io" in the docker console says it's connected to the server I chose, and downloading a test torrent from https://torguard.net/checkmytorrentipaddress.php also reports back the IP of the VPN...so everything seems to be working as it should. Is Wireguard just that much better than OpenVPN? It was certainly using more of my CPU as it was downloading at that rate, so it seems as if it is VPNing properly? Thanks

Yeah I checked the OVPN files too and saw they were same as what you can download right now. I tried a different server too and it didn't make any difference. Hopefully this is something binhex can help resolve! Thanks!

Am in the exact same boat - tried to use qbittorrent but it's all of a sudden doing the same as SentientNut. Am on PIA too. Edit: How can one downgrade as you said you have done? Thanks Edit 2: Figured out how to downgrade, super easy. Is this something that has changed with the the docker or has something changed at PIA you think?

Thank you for the advice. I wiped the drives not long after my original post at the start of January -- getting rid of the ZFS pool and setting it up as BTRFS. Has worked like a charm so far. Hard linking works without issue across shares. I have not stressed tested how much throughput I get with my setup (not sure the best way to do this on Unraid?)...but I should preface, although I am using Samsung 970 EVO Plus drives, which are PCIe 3.0 x4, cause my board is an X470 motherboard with only 2 NVMe slots -- I think because I have 2 of the PCIe slots occupied (GPU & HBA controller), one of the drives is running at PCe 2.0 x4...if I have understood the mobo manual correctly. I would be interested in running a test though to see what the max read/writes of the BTRFS pool would be, if anyone knows? Does anyone also know if there is a way to see what PCIe gen and number of lanes each of the drives are using, so I can confirm? Thanks!

Ok that would make sense, only the log file is the *only* file that I cannot open in the backups. All the files it backs up I can open from a Windows PC using SMB and my normal login credentials.

Hello again, The backup script seems to be working great for me, thank you. The only thing I find is that I am unable to open the log files that are created by the script in each backup folder. I can open them through the Unraid GUI but not from my Windows computer. The log files have the permission "root" and "-rw-------". Is there a way to make it so I can open the log files from my computer? Thank you!

Damn it, Just found this from Spaceinvador one. Even if hardlinking was working within one dataset for me (which it isn't), it doesn't really work for me as I would need it to work across datasets. I think it's probably best for me to abandon ZFS and go with BTRFS.

Hello, I am trying to set up my 2 x 2TB NVMe drives as a cache pool and store my downloads share data on there. I have set up my multimedia share to put new files on the cache drive first, then move to the array. I wanted to be able to: Download to the Downloads share Copy-Paste the downloaded file to it's final destination share on the cache drive...using a hardlink for speed and saving space (no point having the same data twice) Then Unraid copies it to the array, leaving the original download to continue seeding. I have set my drives up as a ZFS mirror pool. Hardlinking does not appear to be working though. I have had hardlinking enabled in the Global Share Settings of Unraid for ages and it works fine on the array, but it is not working on this ZFS pool. I have tried copying to and from in each permutation of standard folder and dataset and regardless, it does a standard copy rather than a hardlink. This is all through Windows Explorer as well btw, like I have always done it. I feel like I am missing something simple? I could understand if i was trying to copy between two pools or two different file systems or something, but this is just basic copying on the same disk. Could really do with a solid on this one. Thanks!