Vm GPU passthrough help


Recommended Posts

Hi all,

 

i'm trying to create VM's with GPU passthrough (Windows and Linux) But on creation i get this error

internal error: process exited while connecting to monitor: 2016-03-30T18:03:01.497204Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to set iommu for container: Operation not permitted
2016-03-30T18:03:01.497229Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to setup container for group 23
2016-03-30T18:03:01.497238Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 23
2016-03-30T18:03:01.497247Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed
2016-03-30T18:03:01.497257Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized

Tried several different video cards and they all do the same.

 

This is my info page

Model: Custom
M/B: ASUSTeK COMPUTER INC. - Crosshair V Formula
CPU: AMD FX(tm)-8350 Eight-Core @ 4000
HVM: Enabled
IOMMU: Enabled
Cache: 384 kB, 8192 kB, 8192 kB
Memory: 32768 MB (max. installable capacity 32 GB)
Network: eth0: 1000Mb/s - Full Duplex
Kernel: Linux 4.1.18-unRAID x86_64

PCI Devices

00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx0 port B) (rev 02)
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port B)
00:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port D)
00:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port E)
00:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port F)
00:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port G)
00:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port H)
00:0c.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890S PCI Express bridge for GPP2 port 1
00:0d.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx1 port B)
00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller (rev 42)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge (rev 40)
00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5
01:00.0 VGA compatible controller: NVIDIA Corporation GF119 [GeForce GT 610] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)
02:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)
03:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
04:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
05:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
06:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
07:00.0 VGA compatible controller: NVIDIA Corporation GT218 [GeForce 210] (rev a2)
07:00.1 Audio device: NVIDIA Corporation High Definition Audio Controller (rev a1)
08:00.0 VGA compatible controller: NVIDIA Corporation GT218 [GeForce 210] (rev a2)
08:00.1 Audio device: NVIDIA Corporation High Definition Audio Controller (rev a1)

IOMMU groups

/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:02.0
/sys/kernel/iommu_groups/2/devices/0000:00:04.0
/sys/kernel/iommu_groups/3/devices/0000:00:05.0
/sys/kernel/iommu_groups/4/devices/0000:00:06.0
/sys/kernel/iommu_groups/5/devices/0000:00:07.0
/sys/kernel/iommu_groups/6/devices/0000:00:09.0
/sys/kernel/iommu_groups/7/devices/0000:00:0c.0
/sys/kernel/iommu_groups/8/devices/0000:00:0d.0
/sys/kernel/iommu_groups/9/devices/0000:00:11.0
/sys/kernel/iommu_groups/10/devices/0000:00:12.0
/sys/kernel/iommu_groups/10/devices/0000:00:12.2
/sys/kernel/iommu_groups/11/devices/0000:00:13.0
/sys/kernel/iommu_groups/11/devices/0000:00:13.2
/sys/kernel/iommu_groups/12/devices/0000:00:14.0
/sys/kernel/iommu_groups/13/devices/0000:00:14.3
/sys/kernel/iommu_groups/14/devices/0000:00:14.4
/sys/kernel/iommu_groups/15/devices/0000:00:14.5
/sys/kernel/iommu_groups/16/devices/0000:00:16.0
/sys/kernel/iommu_groups/16/devices/0000:00:16.2
/sys/kernel/iommu_groups/17/devices/0000:01:00.0
/sys/kernel/iommu_groups/17/devices/0000:01:00.1
/sys/kernel/iommu_groups/18/devices/0000:02:00.0
/sys/kernel/iommu_groups/19/devices/0000:03:00.0
/sys/kernel/iommu_groups/20/devices/0000:04:00.0
/sys/kernel/iommu_groups/21/devices/0000:05:00.0
/sys/kernel/iommu_groups/22/devices/0000:06:00.0
/sys/kernel/iommu_groups/23/devices/0000:07:00.0
/sys/kernel/iommu_groups/23/devices/0000:07:00.1
/sys/kernel/iommu_groups/24/devices/0000:08:00.0
/sys/kernel/iommu_groups/24/devices/0000:08:00.1

 

Any ideas?

 

 

Link to comment

** Bump **

 

I've tried the  Hotfix: Device 'vfio-pci' could not be initialized, Several different video cards, Several different versions of unraid and 2 different mobo's and cpu's with no luck at all. Same error on VM create.

 

internal error: early end of file from monitor: possible problem:
2016-04-02T04:45:14.582543Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to set iommu for container: Operation not permitted
2016-04-02T04:45:14.582576Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to setup container for group 11
2016-04-02T04:45:14.582585Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 11
2016-04-02T04:45:14.582597Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed
2016-04-02T04:45:14.582609Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized

 

I'm not sure what else i can attempt to do... i want to get this up and running because im actually doing a school project on Unraid it's self and i want to show off the power of it.

Link to comment

First tried  it with a nvidia gt 610 first slot,runs unraid all good.

then 2x290x's in the second slot and third slot, didnt work.

So i switched out the the AMD cards and put two GT 210's in still same error,

switched to 1 GT 210 and still getting the same error.

 

So i tried doing this on a different machine, different mobo, different proc.

Had 1 5770 in the top slot that runs unraid all good

and a 7770 in the second slot, couldn't pass that through,

so i moved the 7770 to the bottom and last slot. same deal.

 

Both machines, and all attempts the IOMMU and HVM are enabled. and the video cards paired with their audio are in separate IOMMU groups.

Link to comment

Looking at the syslog shows that your bios is broken when it comes to iommu. That is probably why it's not working.

I do not know how to fix that, but you might want to check if there is a bios update available.

 

On the Asus Crosshair I know the bios is broken as I have the same board. You need to add this to your syslinux.cfg after append somewhere:

ivrs_ioapic[9]=00:14.0 ivrs_ioapic[10]=00:00.1

That is for the Formula Z board, but I think it's the same on both boards. I'll see if I can dig up my syslinux.cfg from when I used that board.

Link to comment

ivrs_ioapic[9]=00:14.0 ivrs_ioapic[10]=00:00.1

 

Worked on my Asus board. i was able to pass through both my GPU's to different VM's. Had a blue screen on driver install Win8 but ill try that again sometime.

 

Will see what i can do with my other build, ill check my bios.

Thanks for your help, it was very useful.

Link to comment

So the one build with the Asus works all good.

 

My personal server im running at home is not so much still. i did a bios update.

Error :

internal error: process exited while connecting to monitor: 2016-04-06T04:17:25.574181Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to set iommu for container: Operation not permitted
2016-04-06T04:17:25.574214Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to setup container for group 11
2016-04-06T04:17:25.574223Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 11
2016-04-06T04:17:25.574234Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed
2016-04-06T04:17:25.574246Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized

 

Info:

Model: Antec
M/B: ASUSTeK Computer INC. - P6T SE
CPU: Intel® Core™ i7 CPU 920 @ 2.67GHz
HVM: Enabled
IOMMU: Enabled
Cache: 256 kB, 1024 kB, 8192 kB
Memory: 12288 MB (max. installable capacity 24 GB)
Network: eth0: 1000Mb/s - Full Duplex
Kernel: Linux 4.1.18-unRAID x86_64
OpenSSL: 1.0.1s
Uptime:

 

PCI Devices:

00:00.0 Host bridge: Intel Corporation 5520/5500/X58 I/O Hub to ESI Port (rev 13)
00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 13)
00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13)
00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 13)
00:14.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub System Management Registers (rev 13)
00:14.1 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 13)
00:14.2 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 13)
00:14.3 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Throttle Registers (rev 13)
00:1a.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
00:1c.2 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 3
00:1c.3 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 4
00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5
00:1d.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller #1
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller #2
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde XT [Radeon HD 7770/8760 / R7 250X]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Juniper XT [Radeon HD 5770]
02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series]
04:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
04:00.1 IDE interface: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
05:00.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6315 Series Firewire Controller
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 02)
ff:00.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture Generic Non-Core Registers (rev 05)
ff:00.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture System Address Decoder (rev 05)
ff:02.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 0 (rev 05)
ff:02.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 0 (rev 05)
ff:03.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller (rev 05)
ff:03.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Target Address Decoder (rev 05)
ff:03.4 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Test Registers (rev 05)
ff:04.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Control Registers (rev 05)
ff:04.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Address Registers (rev 05)
ff:04.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Rank Registers (rev 05)
ff:04.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Thermal Control Registers (rev 05)
ff:05.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Control Registers (rev 05)
ff:05.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Address Registers (rev 05)
ff:05.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Rank Registers (rev 05)
ff:05.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Thermal Control Registers (rev 05)
ff:06.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Control Registers (rev 05)
ff:06.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Address Registers (rev 05)
ff:06.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Rank Registers (rev 05)
ff:06.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Thermal Control Registers (rev 05)

 

IOMMU Groups

/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/2/devices/0000:00:03.0
/sys/kernel/iommu_groups/3/devices/0000:00:07.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.1
/sys/kernel/iommu_groups/4/devices/0000:00:14.2
/sys/kernel/iommu_groups/4/devices/0000:00:14.3
/sys/kernel/iommu_groups/5/devices/0000:00:1a.0
/sys/kernel/iommu_groups/5/devices/0000:00:1a.1
/sys/kernel/iommu_groups/5/devices/0000:00:1a.2
/sys/kernel/iommu_groups/5/devices/0000:00:1a.7
/sys/kernel/iommu_groups/6/devices/0000:00:1b.0
/sys/kernel/iommu_groups/7/devices/0000:00:1c.0
/sys/kernel/iommu_groups/7/devices/0000:00:1c.2
/sys/kernel/iommu_groups/7/devices/0000:00:1c.3
/sys/kernel/iommu_groups/7/devices/0000:00:1c.4
/sys/kernel/iommu_groups/7/devices/0000:04:00.0
/sys/kernel/iommu_groups/7/devices/0000:04:00.1
/sys/kernel/iommu_groups/7/devices/0000:05:00.0
/sys/kernel/iommu_groups/7/devices/0000:06:00.0
/sys/kernel/iommu_groups/8/devices/0000:00:1d.0
/sys/kernel/iommu_groups/8/devices/0000:00:1d.1
/sys/kernel/iommu_groups/8/devices/0000:00:1d.2
/sys/kernel/iommu_groups/8/devices/0000:00:1d.7
/sys/kernel/iommu_groups/9/devices/0000:00:1e.0
/sys/kernel/iommu_groups/10/devices/0000:00:1f.0
/sys/kernel/iommu_groups/10/devices/0000:00:1f.2
/sys/kernel/iommu_groups/10/devices/0000:00:1f.3
/sys/kernel/iommu_groups/10/devices/0000:00:1f.5
/sys/kernel/iommu_groups/11/devices/0000:01:00.0
/sys/kernel/iommu_groups/11/devices/0000:01:00.1
/sys/kernel/iommu_groups/12/devices/0000:02:00.0
/sys/kernel/iommu_groups/12/devices/0000:02:00.1
/sys/kernel/iommu_groups/13/devices/0000:ff:00.0
/sys/kernel/iommu_groups/13/devices/0000:ff:00.1
/sys/kernel/iommu_groups/14/devices/0000:ff:02.0
/sys/kernel/iommu_groups/14/devices/0000:ff:02.1
/sys/kernel/iommu_groups/15/devices/0000:ff:03.0
/sys/kernel/iommu_groups/15/devices/0000:ff:03.1
/sys/kernel/iommu_groups/15/devices/0000:ff:03.4
/sys/kernel/iommu_groups/16/devices/0000:ff:04.0
/sys/kernel/iommu_groups/16/devices/0000:ff:04.1
/sys/kernel/iommu_groups/16/devices/0000:ff:04.2
/sys/kernel/iommu_groups/16/devices/0000:ff:04.3
/sys/kernel/iommu_groups/17/devices/0000:ff:05.0
/sys/kernel/iommu_groups/17/devices/0000:ff:05.1
/sys/kernel/iommu_groups/17/devices/0000:ff:05.2
/sys/kernel/iommu_groups/17/devices/0000:ff:05.3
/sys/kernel/iommu_groups/18/devices/0000:ff:06.0
/sys/kernel/iommu_groups/18/devices/0000:ff:06.1
/sys/kernel/iommu_groups/18/devices/0000:ff:06.2
/sys/kernel/iommu_groups/18/devices/0000:ff:06.3

 

The code you gave me for the syslinux.cfg but it dosn't seem to have an effect

I'll Attach my diagnostics also

unrad-diagnostics-20160405-2322.zip

Link to comment

A quick google search shows alot of broken bios on the board you have, so it might not be possible to use.

Have you tried adding iommu=pt in the syslinux.cfg?

You should remove the fix you added for your other board as that is specific for that board.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.