Jump to content

matthope

Members
  • Posts

    76
  • Joined

  • Last visited

Posts posted by matthope

  1. Hi,

     

    The issue:

     

    At the moment, there is two way to use a 3D application in a VM with unRAID. First way is to passtrough a GPU to it. While this solution is nice and work well if you have only 1 or 2 VM's that need GPU acceleration, it's not ideal when you need multiples accelerated VM's. The second way is to use the intel GVT capabilities, however, the UPT is not super well implemented yet and the graphical power of those IGD are not that great.

     

    The solution:

     

    It would be nice if you could include GPU virtualisation (vGPU) solution, there is more than one solution to achieve this, but I think virtio-gpu and virGL is our best shoot. There is also MxGPU with the AMD GIM driver or Nvidia GRID but it require expensive gpu card to work. 

     

    If virtio-gpu is implemented, it would be nice to see spice implemented as well since its more efficient than a VNC.

     

    Also I might dreaming here, but if there was a way to use one GPU to display more than one VM desktop. Example, if there 3 hdmi output on the card, display the desktop on the VM 1 on the HDM1 and the VM 2 on the HDMI2 and so on. 

     

    Some interestings links :

     

    https://www.spice-space.org/spice-user-manual.html 

    https://www.kraxel.org/blog/2016/09/using-virtio-gpu-with-libvirt-and-spice/

    https://www.kraxel.org/slides/qemu-gfx-2016/

     

    Regards, Mathieu

    • Like 8
    • Upvote 3
  2. I've found a solution in the [email protected] archives, just add those line at the end of the VM XML config. If you're passingtrough more than one device, just make sure to select the hostdev# accordingly to your intel IGD.

     

    [...]  
      <qemu:commandline>
        <qemu:arg value='-set'/>
        <qemu:arg value='device.hostdev0.x-igd-opregion=on'/>
        <qemu:arg value='-set'/>
        <qemu:arg value='device.hostdev0.x-igd-gms=1'/>
      </qemu:commandline>
    </domain>

     

    • Thanks 1
    • Upvote 1
  3. I might have found something interesting from the QEMU Github . It's said that legacy mode require a fw_cfg file, however i've look at the qemu command that unraid auto-generate and it doesn't contains any -fw_cfg parameter. 

    Legacy mode IGD support imposes two fw_cfg requirements on the VM firmware:
    
    1) "etc/igd-opregion"
    
       This fw_cfg file exposes the OpRegion for the IGD device.  A reserved
       region should be created below 4GB (recommended 4KB alignment), sized
       sufficient for the fw_cfg file size, and the content of this file copied
       to it.  The dword based address of this reserved memory region must also
       be written to the ASLS register at offset 0xFC on the IGD device.  It is
       recommended that firmware should make use of this fw_cfg entry for any
       PCI class VGA device with Intel vendor ID.  Multiple of such devices
       within a VM is undefined.
    
    2) "etc/igd-bdsm-size"
    
       This fw_cfg file contains an 8-byte, little endian integer indicating
       the size of the reserved memory region required for IGD stolen memory.
       Firmware must allocate a reserved memory below 4GB with required 1MB
       alignment equal to this size.  Additionally the base address of this
       reserved region must be written to the dword BDSM register in PCI config
       space of the IGD device at offset 0x5C.  As this support is related to
       running the IGD ROM, which has other dependencies on the device appearing
       at guest address 00:02.0, it's expected that this fw_cfg file is only
       relevant to a single PCI class VGA device with Intel vendor ID, appearing
       at PCI bus address 00:02.0.

     

    From the Qemu man page : 

     

    -fw_cfg [name=]name,file=file
    Add named fw_cfg entry with contents from file file.
    -fw_cfg [name=]name,string=str
    Add named fw_cfg entry with contents from string str.
    The terminating NUL character of the contents of str will not be included as part of the fw_cfg item data. To insert contents with embedded NUL characters, you have to use the file parameter.
    
    The fw_cfg entries are passed by QEMU through to the guest.
    
    Example:
    
            -fw_cfg name=opt/com.mycompany/blob,file=./my_blob.bin
    creates an fw_cfg entry named opt/com.mycompany/blob with contents from ./my_blob.bin.

     

    Anybody to help me with this, or at least look at it?

     

  4. Hello everyone,

     

    Recently, I have remarked a bunch of bot entries in my nginx access log file. Before that, fail2ban bot filter seemed to work fine. So I have looked in my fail2ban log file and it's full of error. It is the config by default, I've never modified it. I've tried to update fail2ban and ip6tables, but I cant find any package manager in the docker. I wonder if anyone know how to fix that. Here the error message: 

    2017-09-21 16:13:21,035 fail2ban.server         [261]: INFO    --------------------------------------------------
    2017-09-21 16:13:21,035 fail2ban.server         [261]: INFO    Starting Fail2ban v0.10.0a1
    2017-09-21 16:13:21,035 fail2ban.server         [261]: INFO    Daemon started
    2017-09-21 16:13:21,038 fail2ban.database       [261]: INFO    Connected to fail2ban persistent database '/config/fail2ban/fail2ban.sqlite3'
    2017-09-21 16:13:21,039 fail2ban.jail           [261]: INFO    Creating new jail 'nginx-http-auth'
    2017-09-21 16:13:21,040 fail2ban.jail           [261]: INFO    Jail 'nginx-http-auth' uses poller
    2017-09-21 16:13:21,041 fail2ban.filter         [261]: INFO    Set jail log file encoding to UTF-8
    2017-09-21 16:13:21,041 fail2ban.jail           [261]: INFO    Initiated 'polling' backend
    2017-09-21 16:13:21,042 fail2ban.filter         [261]: INFO    Added logfile = /config/log/nginx/error.log (pos = 7134, hash = e98d121622aabfa4a1a34b1d636c2af5)
    2017-09-21 16:13:21,043 fail2ban.filter         [261]: INFO    Set maxRetry = 5
    2017-09-21 16:13:21,043 fail2ban.filter         [261]: INFO    Set jail log file encoding to UTF-8
    2017-09-21 16:13:21,044 fail2ban.actions        [261]: INFO    Set banTime = 600
    2017-09-21 16:13:21,044 fail2ban.filter         [261]: INFO    Set findtime = 600
    2017-09-21 16:13:21,047 fail2ban.jail           [261]: INFO    Creating new jail 'nginx-botsearch'
    2017-09-21 16:13:21,047 fail2ban.jail           [261]: INFO    Jail 'nginx-botsearch' uses poller
    2017-09-21 16:13:21,047 fail2ban.filter         [261]: INFO    Set jail log file encoding to UTF-8
    2017-09-21 16:13:21,048 fail2ban.jail           [261]: INFO    Initiated 'polling' backend
    2017-09-21 16:13:21,049 fail2ban.filter         [261]: INFO    Added logfile = /config/log/nginx/access.log (pos = 480286, hash = 7cdbb6fa5cd3b6fb68a493f221b06792)
    2017-09-21 16:13:21,049 fail2ban.filter         [261]: INFO    Set maxRetry = 2
    2017-09-21 16:13:21,050 fail2ban.filter         [261]: INFO    Set jail log file encoding to UTF-8
    2017-09-21 16:13:21,050 fail2ban.actions        [261]: INFO    Set banTime = 600
    2017-09-21 16:13:21,050 fail2ban.filter         [261]: INFO    Set findtime = 600
    2017-09-21 16:13:21,054 fail2ban.jail           [261]: INFO    Creating new jail 'nginx-badbots'
    2017-09-21 16:13:21,054 fail2ban.jail           [261]: INFO    Jail 'nginx-badbots' uses poller
    2017-09-21 16:13:21,054 fail2ban.filter         [261]: INFO    Set jail log file encoding to UTF-8
    2017-09-21 16:13:21,054 fail2ban.jail           [261]: INFO    Initiated 'polling' backend
    2017-09-21 16:13:21,055 fail2ban.filter         [261]: INFO    Added logfile = /config/log/nginx/access.log (pos = 480286, hash = 7cdbb6fa5cd3b6fb68a493f221b06792)
    2017-09-21 16:13:21,056 fail2ban.filter         [261]: INFO    Set maxRetry = 2
    2017-09-21 16:13:21,056 fail2ban.filter         [261]: INFO    Set jail log file encoding to UTF-8
    2017-09-21 16:13:21,057 fail2ban.actions        [261]: INFO    Set banTime = 600
    2017-09-21 16:13:21,057 fail2ban.filter         [261]: INFO    Set findtime = 600
    2017-09-21 16:13:21,065 fail2ban.jail           [261]: INFO    Jail 'nginx-http-auth' started
    2017-09-21 16:13:21,066 fail2ban.jail           [261]: INFO    Jail 'nginx-botsearch' started
    2017-09-21 16:13:21,068 fail2ban.jail           [261]: INFO    Jail 'nginx-badbots' started
    2017-09-21 16:13:21,113 fail2ban.utils          [261]: ERROR   ip6tables -w -N f2b-nginx-http-auth
    ip6tables -w -A f2b-nginx-http-auth -j RETURN
    ip6tables -w -I INPUT -p tcp -m multiport --dports http,https -j f2b-nginx-http-auth -- stderr:
    2017-09-21 16:13:21,113 fail2ban.utils          [261]: ERROR    -- stderr: "modprobe: can't change directory to '/lib/modules': No such file or directory"
    2017-09-21 16:13:21,113 fail2ban.utils          [261]: ERROR    -- stderr: "ip6tables v1.6.1: can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?)"
    2017-09-21 16:13:21,113 fail2ban.utils          [261]: ERROR    -- stderr: 'Perhaps ip6tables or your kernel needs to be upgraded.'
    2017-09-21 16:13:21,114 fail2ban.utils          [261]: ERROR    -- stderr: "modprobe: can't change directory to '/lib/modules': No such file or directory"
    2017-09-21 16:13:21,114 fail2ban.utils          [261]: ERROR    -- stderr: "ip6tables v1.6.1: can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?)"
    2017-09-21 16:13:21,114 fail2ban.utils          [261]: ERROR    -- stderr: 'Perhaps ip6tables or your kernel needs to be upgraded.'
    2017-09-21 16:13:21,114 fail2ban.utils          [261]: ERROR    -- stderr: 'Could not open socket to kernel: Address family not supported by protocol'
    2017-09-21 16:13:21,114 fail2ban.utils          [261]: ERROR   ip6tables -w -N f2b-nginx-http-auth
    ip6tables -w -A f2b-nginx-http-auth -j RETURN
    ip6tables -w -I INPUT -p tcp -m multiport --dports http,https -j f2b-nginx-http-auth -- returned 1
    2017-09-21 16:13:21,114 fail2ban.actions        [261]: ERROR   Failed to start jail 'nginx-http-auth' action 'iptables-multiport': Error starting action Jail('nginx-http-auth')/iptables-multiport
    2017-09-21 16:13:21,125 fail2ban.utils          [261]: ERROR   ip6tables -w -N f2b-nginx-botsearch
    ip6tables -w -A f2b-nginx-botsearch -j RETURN
    ip6tables -w -I INPUT -p tcp -m multiport --dports http,https -j f2b-nginx-botsearch -- stderr:
    2017-09-21 16:13:21,126 fail2ban.utils          [261]: ERROR    -- stderr: "modprobe: can't change directory to '/lib/modules': No such file or directory"
    2017-09-21 16:13:21,126 fail2ban.utils          [261]: ERROR    -- stderr: "ip6tables v1.6.1: can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?)"
    2017-09-21 16:13:21,126 fail2ban.utils          [261]: ERROR    -- stderr: 'Perhaps ip6tables or your kernel needs to be upgraded.'
    2017-09-21 16:13:21,126 fail2ban.utils          [261]: ERROR    -- stderr: "modprobe: can't change directory to '/lib/modules': No such file or directory"
    2017-09-21 16:13:21,126 fail2ban.utils          [261]: ERROR    -- stderr: "ip6tables v1.6.1: can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?)"
    2017-09-21 16:13:21,126 fail2ban.utils          [261]: ERROR    -- stderr: 'Perhaps ip6tables or your kernel needs to be upgraded.'
    2017-09-21 16:13:21,126 fail2ban.utils          [261]: ERROR    -- stderr: 'Could not open socket to kernel: Address family not supported by protocol'
    2017-09-21 16:13:21,126 fail2ban.utils          [261]: ERROR   ip6tables -w -N f2b-nginx-botsearch
    ip6tables -w -A f2b-nginx-botsearch -j RETURN
    ip6tables -w -I INPUT -p tcp -m multiport --dports http,https -j f2b-nginx-botsearch -- returned 1
    2017-09-21 16:13:21,126 fail2ban.actions        [261]: ERROR   Failed to start jail 'nginx-botsearch' action 'iptables-multiport': Error starting action Jail('nginx-botsearch')/iptables-multiport
    2017-09-21 16:13:21,138 fail2ban.utils          [261]: ERROR   ip6tables -w -N f2b-nginx-badbots
    ip6tables -w -A f2b-nginx-badbots -j RETURN
    ip6tables -w -I INPUT -p tcp -m multiport --dports http,https -j f2b-nginx-badbots -- stderr:
    2017-09-21 16:13:21,138 fail2ban.utils          [261]: ERROR    -- stderr: "modprobe: can't change directory to '/lib/modules': No such file or directory"
    2017-09-21 16:13:21,138 fail2ban.utils          [261]: ERROR    -- stderr: "ip6tables v1.6.1: can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?)"
    2017-09-21 16:13:21,138 fail2ban.utils          [261]: ERROR    -- stderr: 'Perhaps ip6tables or your kernel needs to be upgraded.'
    2017-09-21 16:13:21,138 fail2ban.utils          [261]: ERROR    -- stderr: "modprobe: can't change directory to '/lib/modules': No such file or directory"
    2017-09-21 16:13:21,138 fail2ban.utils          [261]: ERROR    -- stderr: "ip6tables v1.6.1: can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?)"
    2017-09-21 16:13:21,138 fail2ban.utils          [261]: ERROR    -- stderr: 'Perhaps ip6tables or your kernel needs to be upgraded.'
    2017-09-21 16:13:21,138 fail2ban.utils          [261]: ERROR    -- stderr: 'Could not open socket to kernel: Address family not supported by protocol'
    2017-09-21 16:13:21,139 fail2ban.utils          [261]: ERROR   ip6tables -w -N f2b-nginx-badbots
    ip6tables -w -A f2b-nginx-badbots -j RETURN
    ip6tables -w -I INPUT -p tcp -m multiport --dports http,https -j f2b-nginx-badbots -- returned 1
    2017-09-21 16:13:21,139 fail2ban.actions        [261]: ERROR   Failed to start jail 'nginx-badbots' action 'iptables-multiport': Error starting action Jail('nginx-badbots')/iptables-multiport

    EDIT: Those errors are present since 2017-07-25 at least (no more log after this date).

  5. Issue:

     

    When I passthough the intel IGD to my VM, the task manager report a usage of 5.2Gb at idle. I've tried using more or less RAM and I figure out that under 4Gb of RAM, the IGD wont work (error 43), therefore the idle ram usage is around 1.2Gb. The VM crash when I allocate between 4 and 5.5 Gb of RAM. However, with higher RAM value the idle value stay stable a 5.2~5.3Gb. Since the OS take ~1.2Gb of ram normally, my guess is that the IGD/QEMU does a memalloc of 4Gb or something like this. It's maybe just a parameter somewhere but losing 4Gb of RAM is kind of annoying.

     

    OS:

     

    Windows 10 Pro x64 (fresh install)

    Virtio 0.1.126

    Latest Intel Driver

     

    Hardware:

     

    CPU: Intel G4600

    IGD: Intel HD 630

    MOBO: B150N-GSM

    RAM: 16Gb DDR3

     

    unraid-srv-diagnostics-20170912-2258.zip

  6. In your motherboard bios did you activate VT-D ? 

    What do you mean by not working? No display? VM crash? 

     

    Those error message in the libvirt log are weird. It look like your issue might come from somewhere else. Try to make a new VM with only a VNC and look if those error are still there.

    2017-09-08 21:20:37.673+0000: 14037: warning : qemuDomainObjTaint:3885 : Domain id=1 name='Windows 10' uuid=8c15425b-524a-beb2-7e9c-f1c7cdb00e41 is tainted: high-privileges
    2017-09-08 21:20:37.673+0000: 14037: warning : qemuDomainObjTaint:3885 : Domain id=1 name='Windows 10' uuid=8c15425b-524a-beb2-7e9c-f1c7cdb00e41 is tainted: host-cpu
    2017-09-08 21:20:43.196+0000: 14037: error : x86FeatureInData:780 : internal error: unknown CPU feature __kvm_hv_spinlocks
    2017-09-08 21:20:43.196+0000: 14037: error : x86FeatureInData:780 : internal error: unknown CPU feature __kvm_hv_vendor_id

     

  7. Intent:

    This guide will take you through the process of passing through the integrated audio of the Intel chipset. This methods is tested with a B150 chipset, but it should work with any recent chipset from Intel. I have found this process in the forum, however, my intent is to try to write an easier post to read.

     

    Prerequisites:

     

    • Working VM
    • Unraid 6.3.x / 6.4.x / 6.5.x / 6.6.x
    • Intel CPU and Bios that supports VT-d
    • Intel Motherboard

     

    Guide:

     

    1. Enable Intel VT-D inside your motherboard bios.

     

    2. Open the Web GUI and go to Settings -> VM Manager and click on Advanced View.

     

    3. Enable the PCIe ACS Override setting, try downstream first. (This step might not be required, i've tried with the override disabled and it looks like it working just fine)

     

    4. Go to Tools -> System Devices and look for an audio device. In my case : 

    [8086:a170] 00:1f.3 Audio device: Intel Corporation Sunrise Point-H HD Audio (rev 31)

    Copy everything between [ ] , this is your device ID and you will need that for the next step.

     

    5. Go to Main tab and under Boot Device, click on Flash.

     

    6. Go to your Syslinux configuration and replace the append line under "label unRAID OS" with this one and then click "Apply".

    append pcie_acs_override=downstream vfio-pci.ids=PUT_YOUR_DEVICE_ID_HERE modprobe.blacklist=i2c_i801,i2c_smbus initrd=/bzroot

          Your syslinux configuration should look like this, except maybe your device ID:

    default /syslinux/menu.c32
    menu title Lime Technology, Inc.
    prompt 0
    timeout 50
    label unRAID OS
      menu default
      kernel /bzimage
      append pcie_acs_override=downstream vfio-pci.ids=8086:a170 modprobe.blacklist=i2c_i801,i2c_smbus initrd=/bzroot
    label unRAID OS GUI Mode
      kernel /bzimage
      append pcie_acs_override=downstream initrd=/bzroot,/bzroot-gui
    label unRAID OS Safe Mode (no plugins, no GUI)
      kernel /bzimage
      append pcie_acs_override=downstream initrd=/bzroot unraidsafemode
    label unRAID OS GUI Safe Mode (no plugins)
      kernel /bzimage
      append pcie_acs_override=downstream initrd=/bzroot,/bzroot-gui unraidsafemode
    label Memtest86+
      kernel /memtest

    6. Reboot your server.

     

    7. Now you should have the choice to add your Intel audio in the Sound Card selection when you're editing your VM.

     

    NOTICE: 

    It may require to unplug your server from the AC to make this work, also sometimes from my experience the sound card may just stop working and the only way I have found to make it work again is to shutdown the server and unplug it from the AC.

     

     

     

    • Like 3
  8. 23 hours ago, aptalca said:

     

    Php updates tend to mess that up (they change the structure of those conf files). That's why we decided to internalize them to maintain control over them and update when necessary.

     

    I'm curious as the why you prefer sockets. In terms of performance, *theoretically* socket *may* provide better performance but according user testimonies, people don't notice a difference even on high traffic sites (> millions of hit per day). In any case it's just how nginx communicates with the php processor so it's all internal and there is no discernible effect for the end user or the admin. TCP port is the default in php so we went with that.

     

    If you really want to use your custom php config, you can always map a custom conf file as a volume, but keep in mind that it will likely break when php is updated and we won't provide you with further support ;-) you're on your own

     

    "-v /path/to/your/custom/www.conf:/etc/php7/php-fpm.d/www.conf" make sure you use the one in the container as the base

     

    Thanks for the help, I will update my site-confs settings instead of php settings next time. I always thought that socket where more efficient and reliable than TCP port but after a small research on google it seem like it doesnt really matters.

  9. YES! I fixed it! I was on good track with the dbus-core. It took me a while to figure out that the docker was based on openSUSE. I'm not really familiar with this distro. Finally, I used zypper to update all package and I have added some missings packages. Those command lines should fix the issue about shared librairie:

     

    ##use the command "docker ps" to find your container ID
    sudo docker exec -i -t CONTAINERID /bin/bash
    zypper refresh
    zypper update
    zypper install libdbus-1-3
    exit
    

     

    Thanks for your work on subsonic docker, regards, Mathieu.

     

  10. I did a clean reinstall and It still dont work:

     

    (/subsonic/transcode/ffmpeg) /subsonic/transcode/ffmpeg: error while loading shared libraries: libdbus-1.so.3: cannot open shared object file: No such file or directory

     

    If you can resolve this issue It would be so nice, right now my subsonic android app don't work because the downsampling use the transcoder. My day are rough without my sweet music :( .

     

    Thanks for all.

  11. Interesting. Well thank you for the information, do you happen to have any logs so I can through and see what is happening behind the scenes.

     

    In fact, I don't really have any other log than the subsonic.log which send always the same error.

     

    (/subsonic/transcode/ffmpeg) /subsonic/transcode/ffmpeg: error while loading shared libraries: libdbus-1.so.3: cannot open shared object file: No such file or directory

     

    I have done some experiment inside the subsonic docker and when I tried to execute ffmpeg i've got this error:

     

    ffmpeg: error while loading shared libraries: libavdevice.so.57: cannot open shared object file: No such file or directory

     

    So I've launched ldconfig to make librarie link refresh but it didn't work. I've read on the internet to reinstall dbus-core but I can't do that from the docker.

     

    I'm pretty sure some file/link are missing/broken in the last docker.

     

    Thanks for trying to resolve this issue.

  12. Thanks for your fast replies.

     

    Hmm, I will have to look into it. I use to supply an ffmpeg version but they are doing a lot changes over there since they are going close source. I use to supply a specific ffmpeg to have flac support that might be out of the question now.

     

    Just to mention this problem occur when I activate transcoding, regardless of the format.

     

    Thanks

  13. Since the last update of the docker ffmpeg stop working on my subsonic docker. I use the 6.0 (build a7857c) version on Unraidv6.

     

    [5/26/16 7:15:29 PM EDT]	INFO	TranscodeInputStream	Starting transcoder: /subsonic/transcode/ffmpeg -i /music/Mathieu's music/Les Cowboys Fringants/Break Syndical/01 - En Berne.flac -map 0:0 -b:a 320k -v 0 -f mp3 -
    [5/26/16 7:15:29 PM EDT]	INFO	InputStreamReaderThread	(/subsonic/transcode/ffmpeg) /subsonic/transcode/ffmpeg: error while loading shared libraries: libdbus-1.so.3: cannot open shared object file: No such file or directory
    

     

    Have you an idea how to fix this ?

     

    Thanks

×
×
  • Create New...