mooky

Did you ever figure a fix for this? I've been running up against this too. NFS completely crashes, my only options are reboot the server, or a "better" solution is to just stop and restart the array. If I was to guess, I'd have to say its related to caching as well. I can reproduce the problem at will by copying a few dozen files (a few GB), and then using my Linux desktop file manager (Pop!_OS "files" which I believe is AKA Nautilus) try to delete the files/directories, and hard crash of the NFS. no shares show up after that in GUI or CLI mode. Even if I log into my unRAID box, the shares are borked, again in GUI/web or in CLI mode. [1818073.719919] shfs[11335]: segfault at 10 ip 0000151ab370c4f1 sp 0000151a73ffeba0 error 4 in libfuse3.so.3.10.5[151ab3708000+19000] [1818073.719940] Code: e8 44 c9 ff ff 8b 85 00 01 00 00 85 c0 0f 85 b6 01 00 00 4c 89 f6 48 89 ef 31 db e8 19 dc ff ff 4c 89 ef 45 31 ed 48 8b 40 20 <4c> 8b 70 10 e8 86 c2 ff ff 48 8d 4c 24 18 45 31 c0 31 d2 4c 89 f6

I've been getting some times when my shares just drop over the last while (6.9.2 and 6.10.3 after a recent update 4 days ago), so I started investigating. I found a legit RAM problem (using memtest), that I believe I fixed by modifying BIOS XMP profile settings. After fixing the RAM (ran 3 passes of memtest successfully where previously it failed on first pass), I took the array offline, and ran XFS filesystem checks which didn't reveal anything interesting, put the array back online, and I thought .. ok, everything is okay now. After 16 hours back in production, I just looked at my log and I got the below cut/paste again (I had an identical nfsd not tainted error previous that I didn't save at the time). Now I'm not sure if this is related to shares dropping, I wasn't around actively using my network when this error happened, but thought I'd see. I have NFS turned on in unRAID, and the "Tunable (fuse_remember)" set to 3600, I have more enough RAM, so that's not an issue, I previously had it set even lower to 600. (sitting at roughly 3 of 16 GB RAM used). Let me know if you require anything else for troubleshooting. Maybe this is completely normal occasional network crapping behavior ("not tainted" leads me to believe ... maybe?), and my share problem crapping out is solved since this time despite the error my shares are still there, I just don't know. I don't have enough time with the array back operational yet to have any definitive answer, but thanks in advance. Jun 23 18:54:03 Citadel kernel: ------------[ cut here ]------------ Jun 23 18:54:03 Citadel kernel: nfsd: non-standard errno: -38 Jun 23 18:54:03 Citadel kernel: WARNING: CPU: 1 PID: 8135 at fs/nfsd/nfsproc.c:886 nfserrno+0x4c/0x54 [nfsd] Jun 23 18:54:03 Citadel kernel: Modules linked in: rpcsec_gss_krb5 xt_nat xt_CHECKSUM ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost vhost_iotlb tap veth xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter xfs nfsd auth_rpcgss oid_registry lockd grace sunrpc md_mod nct6775 hwmon_vid ip6table_filter ip6_tables iptable_filter ip_tables x_tables bonding i915 iosf_mbi i2c_algo_bit ttm drm_kms_helper x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel drm kvm intel_gtt crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd wmi_bmof mxm_wmi rapl intel_cstate mpt3sas agpgart i2c_i801 r8169 syscopyarea i2c_smbus sysfillrect input_leds sysimgblt intel_uncore i2c_core led_class realtek fb_sys_fops ahci video raid_class libahci scsi_transport_sas backlight wmi acpi_pad button thermal fan Jun 23 18:54:03 Citadel kernel: CPU: 1 PID: 8135 Comm: nfsd Not tainted 5.15.46-Unraid #1 Jun 23 18:54:03 Citadel kernel: Hardware name: System manufacturer System Product Name/H110M-A/DP, BIOS 4210 07/03/2019 Jun 23 18:54:03 Citadel kernel: RIP: 0010:nfserrno+0x4c/0x54 [nfsd] Jun 23 18:54:03 Citadel kernel: Code: ff c0 48 83 f8 23 75 e1 80 3d 6f 09 05 00 00 41 bc 00 00 00 05 75 15 48 c7 c7 05 61 7b a0 c6 05 59 09 05 00 01 e8 0b a1 07 e1 <0f> 0b 44 89 e0 41 5c c3 48 83 ec 18 31 c9 ba ff 07 00 00 65 48 8b Jun 23 18:54:03 Citadel kernel: RSP: 0018:ffffc9000063fdc0 EFLAGS: 00010282 Jun 23 18:54:03 Citadel kernel: RAX: 0000000000000000 RBX: ffff8881499c0000 RCX: 0000000000000027 Jun 23 18:54:03 Citadel kernel: RDX: 0000000000000003 RSI: ffffc9000063fc48 RDI: ffff888426c5c510 Jun 23 18:54:03 Citadel kernel: RBP: ffff888148b481a0 R08: ffffffff822b4e68 R09: ffffffff82244e48 Jun 23 18:54:03 Citadel kernel: R10: 00007fffffffffff R11: ffffffff8285e075 R12: 0000000005000000 Jun 23 18:54:03 Citadel kernel: R13: ffff888148b48030 R14: ffff8881288dc900 R15: 00000000ffffffda Jun 23 18:54:03 Citadel kernel: FS: 0000000000000000(0000) GS:ffff888426c40000(0000) knlGS:0000000000000000 Jun 23 18:54:03 Citadel kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jun 23 18:54:03 Citadel kernel: CR2: 0000145ff22fc0b0 CR3: 000000000220a003 CR4: 00000000003706e0 Jun 23 18:54:03 Citadel kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jun 23 18:54:03 Citadel kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Jun 23 18:54:03 Citadel kernel: Call Trace: Jun 23 18:54:03 Citadel kernel: <TASK> Jun 23 18:54:03 Citadel kernel: nfsd_link+0x157/0x1a0 [nfsd] Jun 23 18:54:03 Citadel kernel: nfsd4_link+0x20/0x3a [nfsd] Jun 23 18:54:03 Citadel kernel: nfsd4_proc_compound+0x3ce/0x508 [nfsd] Jun 23 18:54:03 Citadel kernel: nfsd_dispatch+0x184/0x21a [nfsd] Jun 23 18:54:03 Citadel kernel: svc_process+0x487/0x69c [sunrpc] Jun 23 18:54:03 Citadel kernel: ? nfsd_svc+0x2bc/0x2bc [nfsd] Jun 23 18:54:03 Citadel kernel: ? nfsd_shutdown_threads+0x77/0x77 [nfsd] Jun 23 18:54:03 Citadel kernel: nfsd+0xee/0x145 [nfsd] Jun 23 18:54:03 Citadel kernel: kthread+0xde/0xe3 Jun 23 18:54:03 Citadel kernel: ? set_kthread_struct+0x32/0x32 Jun 23 18:54:03 Citadel kernel: ret_from_fork+0x22/0x30 Jun 23 18:54:03 Citadel kernel: </TASK> Jun 23 18:54:03 Citadel kernel: ---[ end trace 5432d2cf7b6c0d9f ]--- citadel-diagnostics-20220623-2024.zip

I'm sure its been said before, but there is a bug upgrading from qbittorrent 4.3.9 to 4.40. The default save paths not the least of which, were changed for me from: /data/torrents and /data/incomplete to: /config/qbittorrent/<something> and /config/qbittorrent/<something> (but unchecked as active) Most other settings were reset as well, but not all. The only things that looked to have been still saved were my subdomain webUI bypass and the username & password to log in. I fixed it and so far even after a restart everything is back to normal, but that was a few minutes of annoyance fixing it. So make sure you have a backup to reference. And no, stopping and replacing the backup of your config file and restarting 4.4.0 results in the same wipeout. So you're going to have to fix it manually.

In my setup, I've noticed, the utp (torrent micro transfer protocol) protocol to slow things down remarkably, even with my firewall open to udp and tcp traffic, my vpn, and qbittorrent. So what I've done is just make sure all three are hard-coded to only accept TCP connections and my traffic went from bottlenecking around 1-1.5 MB/sec to hitting 25+ MB/sec (200mbit+), torrent seeds depending.

Just discovered this myself, unRAID 6.9.2 as well. One core, pegged to 100%, htop showing the offending process to be wsdd. Attempting to fix/bypass, but like DanW said, it may not be unRAID's bug, but if there is a more recent package that fixes it, it is your responsibility to test and merge it into unRAID to prevent a race condition inside your product. My CPU had hundreds of hours of time on that one thread until I noticed it just gobbling power and generating heat for no reason. Seems rather wasteful if you ask me.

@dlandon, So the "scripts file" automatically fires always when that particular disk is plugged in, or is there a list of disk events I can use to trigger stuff in this file? Ideally I'd like to have an rclone sync start on plugin of the disk, and then further trigger once a day until I unplug it and offsite it. I'm going to be doing a rotation of 2 different disks so I can finally make good on legitimately accomplishing a proper 3-2-1 backup strategy.

So it's probably been explained, but searching through 232 pages isn't exactly fun, so I'll ask: When I go into "settings" for a partition, I see the mount point (WD-2TB), and share toggle, then below I see: Script File: /boot/config/plugins/unassigned.devices/WD-2TB.sh RUN IN BACKGROUND (toggle off/on) User Script: (undefined) Script Content: Now I see the WD-2TB.sh script is 0 bytes, and empty. What is the point of it? and how does it differ from "User Script" ? I want to have an rclone sync command do a backup to the USB disk daily; I already know the command works when I issue it manually from a command line. Also, how would the "User Scripts" plugin factor in this equation, would it be better than using any of the above, or just a similar alternative and either works?

Hey @binhex, I get the following errors when i run qbittorrentvpn. If I stop the docker, and remove the "perms.txt" file, then relaunch the image, the error goes away for a little while, but then it returns like herpes. Any help to point me in the right direction. I'm running the docker as UID=3501, GID=100, so although the user isn't the standard 99:100 that unraid expects, they are part of the same group, and the permissions look to be fine with a cursory look. Nobody is 99:100 dockoper is 3501:100 When I started up docker: -rwxrwxr-x 1 nobody users 192 Apr 12 00:02 qBittorrent-data.conf* -rwxrwxr-x 1 dockoper users 3661 Apr 12 00:02 qBittorrent.conf* No complaints from docker from when it was started at 12:01am (when I started it) until 8am, from the supervisord.log: 2021-04-12 00:01:46,148 DEBG 'watchdog-script' stdout output: [info] Privoxy process started [info] Waiting for Privoxy process to start listening on port 8118... 2021-04-12 00:01:46,153 DEBG 'watchdog-script' stdout output: [info] Privoxy process listening on port 8118 2021-04-12 08:00:47,292 DEBG 'watchdog-script' stderr output: sed: preserving permissions for ‘/config/qBittorrent/config/sed5mxUuB’: Operation not permitted 2021-04-12 08:01:17,501 DEBG 'watchdog-script' stderr output: sed: preserving permissions for ‘/config/qBittorrent/config/sedxoHbVt’: Operation not permitted 2021-04-12 08:01:47,644 DEBG 'watchdog-script' stderr output: sed: And I have no idea when exactly, but the permission/ownership changed at some point: -rwxrwxr-x 1 nobody users 197 Apr 12 12:36 qBittorrent-data.conf* -rw-rw-rw- 1 nobody users 3660 Apr 12 12:40 qBittorrent.conf

TexasUnraid, with a tightening of the ip_tables for leakage, you now have to do one of two things as outlines on this page, Q26 https://github.com/binhex/documentation/blob/master/docker/faq/vpn.md That should fix connecting to your bittorrent client.

Tell unraid to spin disks down in settings, and then use the "dynamix cache dirs" unraid application. I find this the simplest solution to not have problems between your download client and radarr/sonarr/etc. EDIT: You can also use "split levels" in your unraid share to designate and help steer where things go, as well as included and excluded disks.

Keep it simple. Here's my docker setup: version: '3.7' services: qbittorrentvpn: image: binhex/arch-qbittorrentvpn container_name: qbitvpn volumes: - /mnt/citadel-library:/data - /container-data/qbitvpn:/config environment: - PUID=99 - PGID=100 - TZ=America/Chicago - VPN_ENABLED=yes - VPN_USER=p####### - VPN_PASS=abc123def456 - VPN_PROV=pia - VPN_CLIENT=openvpn - STRICT_PORT_FORWARD=no - ENABLE_PRIVOXY=yes - LAN_NETWORK=192.168.1.0/24 - WEBUI_PORT=25514 - UMASK=000 cap_add: - NET_ADMIN ports: - "6881:6881" - "6881:6881/udp" - "25514:25514" - "8118:8118" restart: unless-stopped And my unraid is setup with a simple "library" share, under which I have the following directory setup: root(library) | +---\incomplete | +---\movies | +---\music | +---\torrents | +---\tvshows All my dockers access my unraid through "data" inside the docker webUI's. The cache is seamless.

Subsequent to my own previous post, I have rolled back my Alpine Linux version and everything works again. It seems there may be an internal problem with a recent version of iptables routing inside docker itself. That would explain the weird behaviour where it would sometimes start and throw errors in some of my docker images with the above errors I already noted, but others would launch, but their UI's would still be unreachable though their web interfaces, even though they were running. It was really weird behaviour. I don't know if its Alpine specific, or a larger docker problem, but something to note if you are having problems and docker is throwing iptable errors.

okay, I know no one here is an expert necessarily in a particular Linux distribution, in this case, Alpine Linux, but I'm coming up short on google-fu trying to get arch-qbittorrentvpn (and other related docker images) working I had a docker-compose that launched correctly and worked on Alpine 3.12 (after installing all the docker related stuff I needed to make it work) I upgraded my Alpine to 3.13, and now, its broken. I even went as far as doing a fresh install in another with a base install of Alpine 3.13.1 to see if the upgrade borked something, and I can report similar errors firing my docker-compose yaml script. I've attached the yaml file and the errors thrown at the command line. Does anyone have any idea where I can begin? Did something fundamentally change between alpine 3.12 and 3.13? docker-compose.yaml docker-compose-errors.txt

Are you able to bring up the webUI ? If so, the default is as I stated. There are 2 settings in the conf file: WebUI\Password_PBKDF2=<HASHED PASSWORD HERE> WebUI\Username=mooky But those are added once you log in through the GUI, and go into options to change it. What did you define as your WEBUI_PORT? Here's my docker-compose yaml file that fires my instance of the docker: services: qbittorrentvpn: image: binhex/arch-qbittorrentvpn container_name: qbitvpn volumes: - /mnt/myfiles:/data - /persiststorage/qbitvpn:/config environment: - PUID=99 - PGID=100 - TZ=America/Chicago - VPN_ENABLED=yes - VPN_USER=p1234567 # change as necessay - VPN_PASS=9dfglkj43G$3 #change as necessary - VPN_PROV=pia #change as necessary - VPN_CLIENT=openvpn - STRICT_PORT_FORWARD=no - ENABLE_PRIVOXY=yes - LAN_NETWORK=192.168.1.0/24 - WEBUI_PORT=29501 - UMASK=000 cap_add: - NET_ADMIN ports: - "29500:6881" - "29500:6881/udp" - "29501:29501" - "8118:8118" restart: unless-stopped

The openVPN portion doesn't have any defaults, you need to have an already paid-for VPN service you subscribe to. The default login/password for qbittorrent however is: login: admin password: adminadmin Change that the first chance you get.

It's ready AFAIK, even people on reddit say that even though its not official that for the most part, barring some edge case weirdness of your particular setup that its good to go for at least a while now. Backup your whole persistent data for radarr dir: tar -zcvf radarr.gz /yourpathto/radarr/ then in my case, in my docker-compose, i simply change the image it pulls image: linuxserver/radarr:nightly Badda-bing, badda-boom, that's it. Worst comes to worst, you blow up the persistent data directory, because some of it will be updated to v3 versions, and untar the 0.2 copy and roll back to 0.2. I'm surprised it works in 0.2 though, it had been broken AF for me, maybe someone backported the fix *shrug*

Just an FYI to everyone if it's not here already. I've tested the most recent version of radarr 3.0.0.4071 - aka ":nightly" (there probably is newer but on Saturday that was the version) and bittorrent 4.3.x aka ":latest" and the problems with importing are fixed. This does NOT apply to radarr 0.2 which apparently will not be fixed for qbittorrent 4.3.x You must stick with qbittorrent 4.2.5 if you are still on radarr 0.2 I assume, but haven't checked that this is also fixed on sonarr since they were both experiencing it and probably pulling the same code fix.

Everything for me seems to work A-OK, except this error as the docker launches, which doesn't seem to affect the launch of rtorrent, rutorrent, openvpn, or privoxy: rtorrent | 2020-10-25 16:48:05,276 DEBG fd 11 closed, stopped monitoring <POutputDispatcher at 139659360098528 for <Subprocess at 139659360097856 with name pyrocore-script in state RUNNING> (stdout)> rtorrent | 2020-10-25 16:48:05,276 DEBG fd 15 closed, stopped monitoring <POutputDispatcher at 139659360219776 for <Subprocess at 139659360097856 with name pyrocore-script in state RUNNING> (stderr)> rtorrent | 2020-10-25 16:48:05,277 INFO exited: pyrocore-script (exit status 0; expected) rtorrent | 2020-10-25 16:48:05,277 DEBG received SIGCHLD indicating a child quit rtorrent | 2020-10-25 16:48:05,311 DEBG 'start-script' stdout output: rtorrent | [info] Default route for container is 172.20.0.1 Is there anywhere I should start at trying to figure it out? FYI, in case anyone was wondering, the docker-compose code: services: rtorrent: image: binhex/arch-rtorrentvpn container_name: rtorrent volumes: - /mnt/citadel-library:/data - /container-data/rtorrent:/config environment: - PUID=99 - PGID=100 - TZ=America/Chicago - VPN_ENABLED=yes - VPN_USER=xxxxxxxx - VPN_PASS=YYYYYYYYYYYYY - VPN_PROV=pia - VPN_CLIENT=openvpn - STRICT_PORT_FORWARD=no - ENABLE_PRIVOXY=yes - ENABLE_AUTODL_IRSSI=yes - ENABLE_RPC2=yes - ENABLE_RPC2_AUTH=yes - ENABLE_WEBUI_AUTH=yes - RPC2_USER=admin - RPC2_PASS=xYxYxYxYxYxY - WEBUI_USER=admin - WEBUI_PASS=xYxYxYxYxYxY - LAN_NETWORK=192.168.1.0/24 - DEBUG=false - PHP_TZ=UTC - UMASK=000 cap_add: - NET_ADMIN ports: - "9080:9080" - "9443:9443" - "8118:8118" restart: unless-stopped

Your ISP must be "Sunday-driving"

Try another server? Maybe your ISP is having routing issues. Yesterday I was getting 25MB/sec from my PIA connection of choice.

Make sure you're on the new network. They don't have many servers left on the old network, dozens (depending on where you are connecting), compared to hundreds for the next gen servers remote us-chicago.privacy.network 1197 (new - 729 servers) remote us-chicago.privateinternetaccess.com 1197 (old - 12 servers) https://www.privateinternetaccess.com/pages/network/

You don't use mover, or you don't use a cache at all? Just curious.

That doesn't instill confidence. I come from a comp sci background, those sorts of things just shouldn't be, cache should be seamless and without errors barring major hardware malfunction, or an inopportune power failure without a UPS. A random 1 in a billion file corruption because of an OS/kernel problem, sure. But it should be RARE, like willy wonka golden ticket rare.

I am downloading to a share with an SSD cache enabled formatted with BTRFS. Just ran a scrub, no errors. The cache is a brand new (literally 2 weeks old) seagate barracuda 500 GB. I'm pretty sure this is a qbittorrent corruption, as what good is a cache/unraid setup if it just corrupts files arbitrarily (barring actual physical hardware problems)? Would hardlink versus copy have anything to do with the corruption? I'm not sure if I should have it set one way or the other. Currently the Share where the files write out to is an array (I use high watermark array) where both the donwloads and final media locations are, with the SSD cache as a buffer. /data/media/tvshows /data/media/movies /data/media/torrents So far unraid reports no errors on the array or cache, and my setup is pretty well stressed tested before i converted it to unraid 2 weeks ago (ran 24 hours of prime95) and another 12+ hours of memtest86. The CPU/mobo/ram are used, the disks are a mix of old and new (but all check out with zero errors) and I got an LSI controller (sata mode) for the spinning rust, with the cache connected directly to the motherboard bridge because I read that cache on an LSI SAS controller isn't an optimal setup. I might go back to qbittorrent (with binhex suggestion to force check option at the end of every DL) because turning off uTP on deluge seems to be a big pain in the butt with a plugin (ltConfig) that isn't updated for a long time, and apparently isn't working exactly 100% (boolean check boxes not working) and I got the same problem someone else reported, even if I manually change it in the config file, it still isn't getting me above the ~1.5MB/sec wall. At least on qbittorrent I was pushing up to 20MB/sec (160Mbit) BTW, Thank you binhex for tracking down the bug on the additional trackers!!! You are awesome!

I'm switching off qbittorrent. I just noticed corruption of files. qbittorrent says a file is 100%, then I watched, and there was a slight, albeit noticeable corruption. So I went went back into qbittorrent, forced a recheck, 99.8% complete. Make a copy of original, restart the "uncompleted" version, it finishes, I "force recheck" one more time just to be certain... 100% ... I sha256 the original and newly fixed file, different hash so obviously something went weird. Plus the newly fixed file no longer has the corruption. I went and checked about 2 dozen other completed torrents, 3 others were also similarly broken. Granted I've got literally several DOZENS of torrents in all the stages of queuing, and downloading, so maybe its got some code that doesn't like tracking that many files at one time. A threading/multitasking problem in the code? With what binhex replied to me above, it just seems qbittorrent, for all its bells and whistles and features, has WAY WAY WAY too many bugs filed. All I really need is a stable download manager that sits in the background and downloads what I tell it to. PS: No, its not my hardware, I've memtest the crap out of it in the last 2 months, its an otherwise stable Xeon box with ECC ram running a few VM's. Going to try binhex/delugevpn