Kaldek

Members
  • Posts

    87
  • Joined

  • Last visited

Posts posted by Kaldek

  1. 13 hours ago, JonathanM said:

    That screenshot shows "Replaceable: Anytime" so it doesn't appear to be a too soon issue.

     

    Have you tried installing and connecting the "Unraid Connect" plugin? I seem to remember something about key management being moved there.


    I have access to all that, and it looks like the image below.  Unfortunately there appears to be no documentation on "Signing Out" a key and what that all means.

     


    image.png.49c1b74513796df56699a0e088211504.png

     

    It drives me up the freaking wall when I find this lack of documentation update.  Documentation must never be delayed and must be part of the release process!  It keeps a high workload on the tech support folks when there's just no need if the damned documentation would be updated.

     

     

  2. I just installed a new DOM-based USB key for my server, following all of the instructions located at https://docs.unraid.net/unraid-os/manual/changing-the-flash-device

     

    There is NO "Replace USB key" option, and it appears this is because I upgraded my license on the 14th of July.  I have had this USB key for over four years.  This is ridiculous, why is a license upgrade classed as a "key replacement"?

     

    image.thumb.png.b5931dbb93577afe685a308ba1a3d380.png

  3. 10 hours ago, diehardbattery said:

    If I corrupt again or the rebuild finishes I'll crack open the case to take a look.  But researching google and asking on the qnap forums didn't turn up anything on it.  Or in the unlikely event that someone else that has the same NAS sees this and volunteers LOL.

    Ah, I seem to have missed the part where you wrote you're running unRAID on a QNAS box.

  4. 19 hours ago, diehardbattery said:

    My NAS already has a DOM with the QNAP OS on it.. wonder if it could be repurposed for Unraid?

    Likely.  Also I'm a bit surprised your motherboard has *no* USB2 headers.  They're common even on new stuff, even it's only a single two-port header.

  5. I burned through a few flash drives before using USB2 ports only.  My current unit has lasted 3 years now.  However, I am switching to a USB DOM (Disk On Module) shortly for some extra reliability. They are more expensive but use quality SLC flash.

     

    The only downside is that it's mounted to a motherboard header, and harder to get to.  But, it should be unlikely I ever need to touch it.

  6. I'm in the middle of some major array disk maneuvering which will require a "New Config" in a few days to remove some drives from the array.  However, I have both a BTRFS cache pool (mirrored 1TB SSDs) and a ZFS RAIDz Pool of 4x 480GB Enterprise grade SSDs.

     

    All of my Appdata and Domains lives on the ZFS pool.  If I lose that, I'm hosed.  So, does the "New Config" option support keeping of both traditional "pools" and also ZFS pools?  It just says "Pool Slots" but doesn't clarify if that will retain ZFS pools.

    image.png.4e90615a7f88ff4237d2a924d0f7de93.png

  7. Hi folks, long time user here.  Upgraded to 6.12 and then 6.12.2 and decided to create a 4-drive SSD ZFS RaidZ pool using some enterprise grade SSDs I was given, and use that pool for all my VMs and Docker containers.

     

    Everything went great, except when I moved the libvirt.img file from the old cache pool to the new zfs pool.  Here's what I did:

    1. Set the system share to use the new ZFS pool
    2. Shut down the Docker engine via Settings-->Docker
    3. "mv /mnt/cache/system/docker /mnt/zfs-cache/system"
    4. Restarted Docker - no issues.
    5. Shut down the VM engine via Settings-->VM Manager
    6. "mv /mnt/cache/system/libvirt /mnt/zfs-cache/system"
    7. Validated that the file exists within /mnt/user/system/libvirt but physically exists only on the ZFS pool
    8. Attempted to restart the VM engine

     

    This gave me "libvirt service failed to start" and the system logs gave me a bunch of errors about btrfs saying that the "file already existed" and information about /dev/loop4 and duplicate entities.  

     

    Quote

    Jul 10 13:13:47 UNRAID emhttpd: shcmd (399): /usr/local/sbin/mount_image '/mnt/user/system/libvirt/libvirt.img' /etc/libvirt 1
    Jul 10 13:13:47 UNRAID kernel: loop4: detected capacity change from 0 to 2097152
    Jul 10 13:13:47 UNRAID root: mount: /etc/libvirt: mount(2) system call failed: File exists.
    Jul 10 13:13:47 UNRAID root:        dmesg(1) may have more information after failed mount system call.
    Jul 10 13:13:47 UNRAID kernel: BTRFS warning: duplicate device /dev/loop4 devid 1 generation 2818 scanned by mount (31598)
    Jul 10 13:13:47 UNRAID root: mount error

     

    The issue went away after the reboot, but, why did it happen in the first place?  I did not have this issue when I moved the docker.img file.


    Diagnostics file also attached.

    unraid-diagnostics-20230710-1317.zip

  8. For what it's worth, here's the code from my discussion with ChatGPT.  This is - as yet - untested.  But knock yourselves out if you want to see what was generated.

    Note that this script is intended to be run "After start of array".

     

    #!/bin/bash
    
    CONTAINER_NAME="frigate"
    
    # Wait for 2 minutes for container to start
    sleep 120
    
    # Get the current container configuration
    CONFIG_JSON=$(docker inspect --type container --format '{{json .}}' ${CONTAINER_NAME})
    
    # Extract the current command and entrypoint
    CMD=$(echo ${CONFIG_JSON} | jq -r '.Config.Cmd | join(" ")')
    ENTRYPOINT=$(echo ${CONFIG_JSON} | jq -r '.Config.Entrypoint | join(" ")')
    
    # Extract all options from the HostConfig property
    HOST_CONFIG_OPTIONS=$(echo ${CONFIG_JSON} | jq -r '.HostConfig | to_entries | map(select(.key != "Devices")) | map("--" + .key + "=\"" + (.value | tostring) + "\"") | join(" ")')
    
    # Replace the USB device option with the new bus ID
    HOST_CONFIG_OPTIONS=$(echo ${HOST_CONFIG_OPTIONS} | sed 's@--device="/dev/bus/usb/004/002@--device="/dev/bus/usb/004/003@g')
    
    # Extract the image name
    IMAGE=$(echo ${CONFIG_JSON} | jq -r '.Config.Image')
    
    # Build the new container run command
    NEW_CMD="docker run --name ${CONTAINER_NAME} ${HOST_CONFIG_OPTIONS} ${ENTRYPOINT} ${CMD}"
    
    # Stop the existing container
    docker stop ${CONTAINER_NAME}
    
    # Run the container with the new configuration
    eval ${NEW_CMD}

     

  9. 17 minutes ago, MrGrey said:

    I've heard a lot about ChatGPT. Is it true?

    Depends what the question is but yes, it's amazing for turning ideas into code.  I don't trust it 100% of course, and I'm using it to give me ideas and examples.  I get to  bypass all the grief I'd get by asking a human. 

    In my view, these generative AI models are necessary.  The amount of time we all burn on questions when the respondee of the question has their own emotions around the question and how they want to answer it is utterly insane.  ChatGPT in particular has a very simple, concise and objective response to everything asked of it.  The trick is knowing how to phrase your questions, hence the term "prompt engineering".  I'm much better at this than I ever was at "Google-Fu".

  10. So folks my unRAID server has an Intel server NIC in it with dual XFP ports and runs the ixgbe driver.  Over a few uNRAID revisions now, there are random instances where the NIC driver dies (and yes, I uploaded the diagnostics files when it happened).  There's really been no solution to this issue, and it of course usually happens when I'm overseas for work and it's 12+ hours before I can remote in to PikVM and bounce the server to get the network back up.

     

    I got a little tired of this, so here's the result of me and ChatGPT4 having a bit of a discussion about how to deal with it automatically.  The solution documented here is a pair of User scripts, one a "Ping Watchdog" and the other a supervisor for the watchdog (in case the watchdog dies).

    Here is the watchdog script, called "ping_watchdog" and is running a ping against a pair of IP addresses (my core switch and my gateway) so that one single IP being down doesn't trigger the reboot.  Sometimes my gateway is off the air for a while as I do some arcane Mikrotik things on it.

    This script is set to run at the first array start only and stays running forever (unless it dies for some reason; see the supervisor script below).
     

    #!/bin/bash
    
    TARGET_IP_1="192.168.0.254" # Replace with your gateway router IP address
    TARGET_IP_2="192.168.0.240" # Replace with your core switch IP address
    PING_COUNT=4 # Number of pings to send
    PING_TIMEOUT=5 # Timeout for each ping in seconds
    FAIL_THRESHOLD=30 # Number of consecutive failed ping checks before restarting
    CHECK_INTERVAL=60 # Time in seconds between ping checks
    
    failed_pings=0
    
    ping_check() {
      local target_ip=$1
      ping -c $PING_COUNT -W $PING_TIMEOUT $target_ip >/dev/null 2>&1
      return $?
    }
    
    while true; do
      ping_check $TARGET_IP_1
      result1=$?
    
      ping_check $TARGET_IP_2
      result2=$?
    
      if [ $result1 -ne 0 ] && [ $result2 -ne 0 ]; then
        failed_pings=$((failed_pings + 1))
        echo "$(date) - Pings to $TARGET_IP_1 and $TARGET_IP_2 failed. Consecutive failed ping checks: $failed_pings"
      else
        failed_pings=0
      fi
    
      if [ $failed_pings -ge $FAIL_THRESHOLD ]; then
        echo "$(date) - Restarting unRAID server due to $FAIL_THRESHOLD consecutive failed ping checks"
        /usr/local/sbin/powerdown -r
        exit 0
      fi
    
      sleep $CHECK_INTERVAL # Wait for the specified time before the next iteration
    done
    


    Next is the "ping_watchdog_supervisor" which is set to run every hour.  If the first script is seen as not running, it kicks it off again.

     

    #!/bin/bash
    
    PING_WATCHDOG_SCRIPT="ping_watchdog"
    
    pid=$(pgrep -f "^/bin/bash.*/tmp/user.scripts/tmpScripts/$PING_WATCHDOG_SCRIPT")
    
    if [ -z "$pid" ]; then
      echo "$(date) - Ping watchdog script not running. Restarting..."
      /usr/local/emhttp/plugins/user.scripts/start_script.sh "$PING_WATCHDOG_SCRIPT"
    else
      echo "$(date) - Ping watchdog script running with PID $pid"
    fi
    

     

    Coupled together, these two scripts ensure that if my NIC ever dies, uNRAID performs a clean reboot without hurting the array.

     

  11. Hi folks.  About to upgrade to Frigate 0.12.0 in case that answers this but I have an issue with my Frigate install on unRAID.

    That is, my Coral TPU busID changes after Frigate loads the Tensor code into it.  At boot time this device is at /dev/bus/usb/004/003 but it then flips to /dev/bus/usb/004/002 after Frigate loads the Tensor software into it.

     

    I'll be damned if I'm going to run this container as privileged and expose all of /dev/bus/usb to it, just to deal with that issue.  Right now I'm in the middle of a rather long chat with GPT4 about how we can get around this issue (currently, dumping the JSON config of the running container, finding the BusID, changing it and then restarting the container with the Google-ified BusID using the same JSON code method).


    Total pain in the ass of course just to flip a couple of bits and then restart a container.

     

    Don't suppose 0.12.0 resolves that issue with the Coral TPU by any chance?

  12. Hi mate, I have set up the Docker container from the official repo and it's working well, with a few items that I suspect you are a good source to discuss them with:

    1. Access via the tunnel to the myunraid.net URL does not work unless I set TLS to "Yes" rather than "strict" so that it uses the self-signed certificate (and I set TLS verification to off in the Cloudflare portal
    2. What is the correct setup if the internal host is accessed via DNS (e.g. host.mydomain.local) rather than IP address?

    It's literally day 1 here so these are questions I would probably be able to work out later anyway.  Figured it can't hurt to ask.

  13. On 5/6/2021 at 3:30 AM, Squid said:

    My response, not the official Limetech response.  Every single user of CA has had to acknowledge two distinct popups regarding applications prior to any installation being allowed.  A general one and also another specifically geared towards plugins, which link to https://forums.unraid.net/topic/87144-ca-application-policies-notes/?tab=comments#comment-809115 and https://forums.unraid.net/topic/87144-ca-application-policies-notes/?tab=comments#comment-817555 respectively, which also detail some (but nowhere near all) of the security procedures in place.

     


    Perhaps the better question is whether Limetech are aware they are actively being probed to find a way in.  It happened to Solarwinds, it can happen here.  It's just way too easy to do this stuff and is asymmetric warfare.  We get one chance to detect the breach whilst the attacker has unlimited time to perfect it by finding even the remotest weakness and using that as the beachhead.

  14. As someone whose full time job is InfoSec for a multinational, I need to reply here.

    Saying not to expose stuff to the Internet is obvious but it doesn't remove the problem.  The biggest concern I have with unRAID is a supply chain attack, and unRAID is popular enough now that e-Crime actors will already be looking at targeting the platform.

    The question is this:  who validates all the plugins etc to make sure they don't have malicious code in them or the ability to subsequently download malicious payloads?  Never mind Docker or VMs, those are the responsibility of the end user - period.   The risk is going to come from the commonly used plugins that everyone uses:

    • Unassigned Devices
    • Nvidia
    • Fix Common Problems
    • Nerd Tools
    • Preclear disks
    • etc

    So, what's Limetech's response?  Better to get ahead of this before it actually happens.   if the risk lies with the end user for 3rd party plugins the link to this in customer agreements need to be pointed out.

    • Like 2
  15. So folks I'm also trying to do this on ESXi v7.0.0 and for some reason I cannot for the life of me send any kind of large data transfer to the ESXi server.  Anything at high speed just dies.  I can open SSH sessions, I can browse to the server, but as soon as I try to push a VMDK file or anything big the write speeds just halt and the network disconnects me.

    Anyone got any ideas here?  This is using vmxnet3 NIC for management.  It is definitely not a storage issue as I have tried multiple datastores - some are virtual images on the unRAID cache drive, some are virtual images on the array.  So, it's definitely a network issue.

    Update: So, I can push data via IPv6 but not IPv4?  Huh?  More investigation required...