grumpybutfun Posted December 11, 2013 Share Posted December 11, 2013 On another point too that I saw in the Xen Arch wiki while doing the network, it says to edit grub to allocate RAM to dom0 as part of the hypervisor installation. Is this still needed or does the installation handle it? I would check what my grub.conf looks like, but since the network is currently gone, that proves difficult... One thing at a time. Yes your network cards look right. It is suppose to use those strange names and not eth0 and eth1. Most Linux Distros are moving away from that. CentOS, Arch, Red Hat, Fedora, etc. do the same thing. Post your /etc/netctl/xenbr0 and whatever else you have in there and I will tell you what they need to be. Quote Link to comment
smakovits Posted December 12, 2013 Share Posted December 12, 2013 ok, so it seems I broke something to a certain extent, but surely the fix is easy... I was using the Xen Arch wiki and things were good and then broke at reboot, sort of. I will post the URL to the wiki as opposed to copying and pasting everything. I started here: https://wiki.archlinux.org/index.php/xen#Modification_of_the_bootloader I went to the point about syslinux users and then skipped to here: https://wiki.archlinux.org/index.php/xen#Creation_of_a_network_bridge things went in well and everything was good until reboot. Upon reboot, enp1s5 was down and xenbr0 had no ip so I rebooted again. This time enp1s5 was up and xenbr0 had an IP address but enp1s5 did not. So, essentially I cannot ssh into the system any more. The system itself is booted and I have console access, but the question is what went wrong. I assume this might have something to do with your mention of a new MAC address since it seems enp1s5 and xenbr0 are the same, maybe. Quote Link to comment
grumpybutfun Posted December 12, 2013 Share Posted December 12, 2013 ok, so it seems I broke something to a certain extent, but surely the fix is easy... I will post the URL to the wiki as opposed to copying and pasting everything. That doesn't help me. I already know how to do it and how to fix whatever problem you have. I need you to copy and paste your xenbr0 and enp1s5 netctl profiles so I can tell you exactly what to put in them to make it work correctly. Quote Link to comment
smakovits Posted December 12, 2013 Share Posted December 12, 2013 Connection=ethernet Description='/etc/netctl/enp1s5-ethernet generated by archboot setup' Interface=enp1s5 IP=dhcp Description="Xen Bridge Connection" Interface=xenbr0 Connection=bridge BindsToInterfaces=(enp1s5) IP=dhcp ## Ignore ®STP and immediately activate the bridge #SkipForwardingDelay=yes Quote Link to comment
grumpybutfun Posted December 12, 2013 Share Posted December 12, 2013 Here is what they should look like: enp1s5 Connection=ethernet Description='/etc/netctl/enp1s5-ethernet generated by archboot setup' Interface=enp1s5 IP=no <--- Make it no xenbr0 Description="xenbr0 Bridge connection" Interface=xenbr0 Connection=bridge BindsToInterfaces=(enp1s5) IP=dhcp ## Ignore ®STP and immediately activate the bridge #SkipForwardingDelay=yes I want you to type in the following commands. You may or may not get an error message (which is okay). I want to clean out your netctl services that start at boot. netctl disable enp1s5-ethernet netctl disable xenbr0 <--- Input the full name of what you call your bridge service. If it's xenbr0-ethernet, enter that. Enable ONLY xenbr0 netctl enable xenbr0 reboot You do not need to enable or start enp6s0-ethernet. Xenbr0 knows what to do. It should be the only network device that is started and enabled. enp1s5 WILL NOT show / get an IP address. Only Xenbr0 will. Lastly, do not go and mess with your grub and other things until you get your Xen working. Once you get it working, Ironic and I will help you tweak and customize it to get the best performance. Quote Link to comment
smakovits Posted December 12, 2013 Share Posted December 12, 2013 just confirming enp6s0 vs enp1s5 Quote Link to comment
grumpybutfun Posted December 12, 2013 Share Posted December 12, 2013 just confirming enp6s0 vs enp1s5 My bad. Make it enp1s5. Good Catch by the way. I updated my post. Quote Link to comment
smakovits Posted December 12, 2013 Share Posted December 12, 2013 ok, got it, bridge created, rebooted and online Quote Link to comment
grumpybutfun Posted December 12, 2013 Share Posted December 12, 2013 ok, got it, bridge created, rebooted and online What does "sudo xl info" show? Quote Link to comment
smakovits Posted December 12, 2013 Share Posted December 12, 2013 testARCH ~ # sudo xl info host : testARCH release : 3.12.2-1-ARCH version : #1 SMP PREEMPT Fri Nov 29 21:14:15 CET 2013 machine : x86_64 nr_cpus : 2 max_cpu_id : 3 nr_nodes : 1 cores_per_socket : 2 threads_per_core : 1 cpu_mhz : 2133 hw_caps : bfebfbff:20100800:00000000:00000900:0000e3bd:00000000:00000001:00000000 virt_caps : hvm total_memory : 4060 free_memory : 3497 sharing_freed_memory : 0 sharing_used_memory : 0 outstanding_claims : 0 free_cpus : 0 xen_major : 4 xen_minor : 3 xen_extra : .1 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : xen_commandline : /xen-4.3.1.gz dom0_mem=512M xsave=1 cc_compiler : gcc (GCC) 4.8.2 cc_compile_by : bige cc_compile_domain : localdomain cc_compile_date : Sun Dec 1 18:22:57 EST 2013 xend_config_format : 4 Quote Link to comment
grumpybutfun Posted December 12, 2013 Share Posted December 12, 2013 testARCH ~ # sudo xl info host : testARCH release : 3.12.2-1-ARCH version : #1 SMP PREEMPT Fri Nov 29 21:14:15 CET 2013 machine : x86_64 nr_cpus : 2 max_cpu_id : 3 nr_nodes : 1 cores_per_socket : 2 threads_per_core : 1 cpu_mhz : 2133 hw_caps : bfebfbff:20100800:00000000:00000900:0000e3bd:00000000:00000001:00000000 virt_caps : hvm total_memory : 4060 free_memory : 3497 sharing_freed_memory : 0 sharing_used_memory : 0 outstanding_claims : 0 free_cpus : 0 xen_major : 4 xen_minor : 3 xen_extra : .1 Congratulations! You should feel proud. I consider myself a Linux Pro and it took me a day or so to figure out how to install Xen and configure it on Arch. Once you go through it, reloading and doing it again is a no brainer. Also, the same way you did it in Arch is also the same way you do it in CentOS, Fedora, openSUSE, etc. All the files and config files are in the exact same place on those other Linux Distros too. Quote Link to comment
smakovits Posted December 12, 2013 Share Posted December 12, 2013 So it is time to proceed to vm creation? It took me a few days still, but overall it was painless with your direction and the much talked about arch Xen wiki. The next steps will likely be fun too. I see the XL conf file can this be named anything? Also, do I copy it to a new location? These will be the little things in my next steps and understand the need for vnc and such, some of the specific settings in the configuration file is where I will have the most questions. Quote Link to comment
grumpybutfun Posted December 12, 2013 Share Posted December 12, 2013 So it is time to proceed to vm creation? It took me a few days still, but overall it was painless with your direction and the much talked about arch Xen wiki. The next steps will likely be fun too. I see the XL conf file can this be named anything? Also, do I copy it to a new location? These will be the little things in my next steps and understand the need for vnc and such, some of the specific settings in the configuration file is where I will have the most questions. Copy and Paste and the configure the file below to your system. Save it in your home directory. Example unRAID.xml file: # Set the name of what you want to call the VM and the number of CPUs and memory # If you offload all your plugins to other VMs and unRAID is back to only be a NAS... It doesn't need much. name = 'unRAID' vcpus = '1' memory = '1024' # Network Configuration is below. # Did you run "ifconfig" and verify the Network Bridge was working and configured? # [u]You need to enter a custom mac address below. It should ALWAYS start with 00:16:3e[/u]. Enter letters in LOWER CASE. # Example of a valid mac address: 00:16:3e:0c:39:24 vif = [ 'mac=00:16:3e:0c:39:22,bridge=xenbr0' ] # If you are assigning unRAID to it's own separate NIC (xenbr1 for example): # vif = [ 'mac=00:16:3e:xx:xx:xx,bridge=xenbr1' ] # The unRAID bzroot and bzimage file need to have the correct path. # It should be the username that you login with. kernel = "/root/VMs/unRAID/bzimage" ramdisk = "/root/VMs/unRAID/bzroot" extra = "console=hvc0" # Extra VM Paramaters: acpi = 1 apic = 1 # Set the following for VNC. Note: Some VNC viewers require a password. #vnc = '1' #vnclisten = '0.0.0.0' #vncpasswd = '' # Set the following for your PCI Device ID(s) you are passingthrough. #pci = [ '00:02.0' ] # If you are passingthrough more than one device it should look like the following: #pci = [ '00:11.0','00:12.2' ] on_xend_stop = 'shutdown' on_xend_start = 'start' To start unRAID xl create unRAID.xml -c The -c command above takes you straight into the unRAID console and you should see it go through the boot process. Once we get unRAID working, I will show you how to autostart unRAID when you reboot or power on your PC. To get back to the Arch console CTL+] Will take you back to the Arch console. To go back into the unRAID console: xl console unRAID (or whatever you called your VM) Quote Link to comment
ironicbadger Posted December 12, 2013 Author Share Posted December 12, 2013 great to see someone else cracking on with this! Quote Link to comment
smakovits Posted December 12, 2013 Share Posted December 12, 2013 So, should I download one of the pre-compiled vhd files? I mean bzimage and bzroot are obvious since they are the normal unRAID files, but all the reading says I need to compile and such or else download an already compiled one great to see someone else cracking on with this! we did sort of take over your thread...but really, I think most everything will help the next guy do most everything, so we are assisting with an impromptu guide if you will Quote Link to comment
grumpybutfun Posted December 12, 2013 Share Posted December 12, 2013 So, should I download one of the pre-compiled vhd files? I mean bzimage and bzroot are obvious since they are the normal unRAID files, but all the reading says I need to compile and such or else download an already compiled one I will send you a link for the bzroot and bzimage. Ironic needs it too. Just sent you a PM. Quote Link to comment
smakovits Posted December 13, 2013 Share Posted December 13, 2013 sweet, I am in. waiting for /dev/disk/by-label/UNRAID (will check for 30 sec)... /dev/disk/by-label/UNRAID not found! Mounting non-root local filesystems: mount: special device /dev/disk/by-label/UNRAID does not exist INIT: Entering runlevel: 3 Going multiuser... Updating shared library links: /sbin/ldconfig & Starting sysklogd daemons: /usr/sbin/syslogd -m0 /usr/sbin/klogd -c 3 -x dhcpcd[726]: version 5.2.7 starting dhcpcd[726]: forked to background, child pid 727 Starting RPC portmapper: /sbin/rpc.portmap Starting RPC NSM (Network Status Monitor): /sbin/rpc.statd Starting Internet super-server daemon: /usr/sbin/inetd Starting system message bus: /usr/bin/dbus-uuidgen --ensure ; /usr/bin/dbus-daemon --system Welcome to Linux 3.9.11p-unRAID (hvc0) Tower login: sure there is more to do, but at least we know it works. I assume this is where I still need to pass-through my USB stick with the rest of the unraid files that actually mean and do something. The is probably the UNRAID disk that it says it cannot fine Quote Link to comment
grumpybutfun Posted December 13, 2013 Share Posted December 13, 2013 I assume this is where I still need to pass-through my USB stick with the rest of the unraid files that actually mean and do something. The is probably the UNRAID disk that it says it cannot fine Correct. Follow Ironic's guide on doing the passthrough and if you have any questions / issues... Post here. Quote Link to comment
smakovits Posted December 13, 2013 Share Posted December 13, 2013 Gotta verify if I can even do pass through with this system. I was only using it to test before blowing up my production system... Quote Link to comment
grumpybutfun Posted December 13, 2013 Share Posted December 13, 2013 Gotta verify if I can even do pass through with this system. I was only using it to test before blowing up my production system... Install Arch to a Flash Drive and when you get that done. Plug it in your Production, boot of it and see if unRAID works (reading the unRAID USB and passing through your drives). You could test everything out before you decide to switch over. Or you could just run Arch off a flash drive and install a data store disk that isn't passed through where your VMs run from. Quote Link to comment
smakovits Posted December 13, 2013 Share Posted December 13, 2013 Gotta verify if I can even do pass through with this system. I was only using it to test before blowing up my production system... Install Arch to a Flash Drive and when you get that done. Plug it in your Production, boot of it and see if unRAID works (reading the unRAID USB and passing through your drives). You could test everything out before you decide to switch over. Or you could just run Arch off a flash drive and install a data store disk that isn't passed through where your VMs run from. Not to sound uninformed, but I thought you cannot run arch from flash, or that it at least does not like it. Also, how much IO is there, for the purpose of the life of the flash. Not that I would, but having the option is tempting. --- Now, lets have some fun. As we are all learning and when I say all, I mean me and the rest of the people following this thread. So, this is fun and we are learning, so I wanted to discuss some concept to all of this, to help better understand the moving parts and how everything comes together. For the purpose of this, I took the supplied unRAID.xml and removed all the comments. What I will do is post your xml and the xl config ironic has on his site for win8, I will mark the similarities and then proceed with discussion. If it gets long and messy we can break it down, but here it goes. unRAID.xml: name = 'unRAID' vcpus = '1' memory = '1024' vif = [ 'mac=00:16:3e:0c:39:22,bridge=xenbr0' ] kernel = "/root/VMs/unRAID/bzimage" ramdisk = "/root/VMs/unRAID/bzroot" extra = "console=hvc0" acpi = 1 apic = 1 #vnc = '1' #vnclisten = '0.0.0.0' #vncpasswd = '' #pci = [ '00:11.0','00:12.2' ] on_xend_stop = 'shutdown' on_xend_start = 'start' Then, ironic's win8 xl config: name = 'windows' vcpus = '4' memory = '8192' vif = [ 'mac=00:16:3E:51:20:4C,bridge=xenbr0,model=e1000' ] kernel = 'hvmloader' disk = [ 'phy:/dev/vg2/win,hda,w', 'phy:/dev/vg1/winDATA,hdb,w', 'file:/data/Windows8Pro.iso,hdc:cdrom,r' ] acpi = '1' apic = '1' vnc = '1' vnclisten = '0.0.0.0' vncpasswd = '' pci = ['01:00.0','01:00.1','00:1d.0'] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' boot = 'dc' viridian = '1' xen_platform_pci='1' sdl = '0' stdvga = '0' usb = '1' usbdevice = 'tablet' builder = 'hvm' device_model_version="qemu-xen-traditional" --- So, right off the bat, we see the kernel being completely different. While not getting it exactly, it seem hvm is needed for windows, to emulate bios and such, but the bzroot is a little less clear, I just assume this is the standalone OS portion of unRAID, so thats why... ramdisk for unraid, is unraid run in RAM? Maybe? extra - reading the syntax guide this just means to take over the console session? I assume the reason this cannot be done with win8 is because win8 uses a full on GUI. on_xend is not in the guide, so I wonder why it is different than the win8 used commands. as far as the disk information for win8, this is pretty clear, load these disks for win8 (c, d, cdrom) boot=[c|d|n] Selects the emulated virtual device to boot from. Options are hard disk ©, cd-rom (d) or network/PXE (n). Multiple options can be given and will be attempted in the order they are given. e.g. to boot from cd-rom but fallback to the hard disk you can give dc. The default is cd. --- This one amuses me, d is network/PXE, but if you do dc, then it is cdrom and then disk. So, what is the option for disk only? viridian, adds vm improvements in win vista and up, 1 means on....turn it on! xen_platform_pci, once again, turn it on as it makes devices and drivers perform better. sdl - Specifies that the display should be presented via an X window (using Simple DirectMedia Layer). The default is not to enable this mode. ---Although I dont quite get it. Is there a case where we would say yes? stdvga - Select a standard VGA card with VBE (VESA BIOS Extensions) as the emulated graphics device. The default is false which means to emulate a Cirrus Logic GD5446 VGA card. If your guest supports VBE 2.0 or later (e.g. Windows XP onwards) then you should enable this. stdvga supports more video ram and bigger resolutions than Cirrus. ---The explanation here tells me we want this to be turned on, but then again, I assume it doesnt matter as this is only used at install. Once we pass-through our real video card, this no longer comes into play. USB - Enables or disables a USB bus in the guest. ---but what does this mean for us? I ask because unRAID whick relies on USB does not have this line, so what does this do for us here? usbdevice=DEVICE Adds DEVICE to the USB bus. The USB bus must also be enabled using usb=1. The most common use for this option is usbdevice=tablet which adds pointer device using absolute coordinates. Such devices function better than relative coordinate devices (such as a standard mouse) since many methods of exporting guest graphics (such as VNC) work better in this mode. Note that this is independent of the actual pointer device you are using on the host/client side. XXX should/could be a list of devices. ---makes sense, this is simply needed for VNC, so I guess the line above is so we can use this to make VNC run well for our install... builder="hvm" Specifies that this is to be an HVM domain. That is, a fully virtualised computer with emulated BIOS, disk and network peripherals, etc. The default is a PV domain, suitable for hosting Xen-aware guest operating systems. ---makes sense, but then takes me back to an item at the top about kernel (kernel = 'hvmloader') as I cannot find this in the guide, just, kernel="PATHNAME" Load the specified file as the kernel image. Either kernel or bootloader must be specified for PV guests. And lastly, device_model_version="DEVICE-MODEL" Selects which variant of the device-model should be used for this guest. Valid values are: qemu-xen-traditional Use the device-model based upon the historical Xen fork of Qemu. This device-model is currently the default. qemu-xen use the device-model merged into the upstream Qemu project. This device-model will become the default in a future version of Xen. It is recommended to accept the default value for new guests. If you have existing guests then, depending on the nature of the guest Operating System, you may wish to force them to use the device model which they were installed with. ---so here I ask why wouldnt we use qemu-xen, if it is the future? --------- So, thats just those two. clear as mud, but it makes some sense, decent sense to a point. some things are there just for vnc to let us get the OS installed. Others are there to make the hardware recognizable by drivers, so it does make sense to that point. The place I still wonder is about your xml vs his config or are the two technically interchangeable, meaning if gave both hypothetical names in this case, they would be unraid.xml and win8.xml and both would live in the home directory. If I am on the right path, are they both also started the same with xl create unRAID.xml -c and xl create win8.xml -c? That seems to be the idea anyway... - After posting, I found this answer to my question above on ironic's blog: You must make sure that you use traiditional qemu as upstream is broken and it will not currently work! It’s in my config so if you just copied and pasted, you’re good to go! One other item of note is ironics specification of NIC model, is that needed? Quote Link to comment
smakovits Posted December 13, 2013 Share Posted December 13, 2013 but, then again, maybe things are not so easy... testARCH /home # xl create win7.xml -c Parsing config from win7.xml WARNING: ignoring "kernel" directive for HVM guest. Use "firmware_override" instead if you really want a non-default firmware libxl: error: libxl_device.c:274:libxl__device_disk_set_backend: Disk vdev=hdc failed to stat: /data/Windows8Pro.iso: No such file or directory libxl: error: libxl_dm.c:1300:libxl__destroy_device_model: could not find device-model's pid for dom 10 libxl: error: libxl.c:1416:libxl__destroy_domid: libxl__destroy_device_model failed for 10 kernel = 'hvmloader' builder = 'hvm' vcpus = '1' memory = '2048' device_model_version="qemu-xen-traditional" disk = [ 'phy:/dev/VMs/win7,hda,w', 'file:/data/Windows8Pro.iso,hdc:cdrom,r' ] name = 'windows' vif = [ 'mac=00:16:3E:51:20:4C,bridge=xenbr0,model=e1000' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' boot = 'dc' acpi = '1' apic = '1' viridian = '1' xen_platform_pci='1' Quote Link to comment
ironicbadger Posted December 13, 2013 Author Share Posted December 13, 2013 It can't find your iso file. Check the paths and remove kernel line, is not needed hence the error message. Will reply in more detail later. Sent from my Nexus 5 using Tapatalk Quote Link to comment
grumpybutfun Posted December 13, 2013 Share Posted December 13, 2013 Not to sound uninformed, but I thought you cannot run arch from flash, or that it at least does not like it. Also, how much IO is there, for the purpose of the life of the flash. Not that I would, but having the option is tempting. Of course you can run Arch on a flash drive. I can't think of a Linux Distro or a OS that can't be run from a Flash Drive. I have done it in Linux, Windows and OSX. Using Arch as an example, it loads the Kernel and your Apps into memory. As far as writing to a flash drive... Those are going to be your log files mostly unless you are running an app that writes a lot of things to your Flash Drive. As in with the Plugins, you could always have those Apps write elsewhere like the cache drive or the RAID itself. You can have the log files write to a ramfs. Which is what unRAID does. In Enterprise Server environments it is common that they either load the OS via Flash Drive or PXE. They have "log" servers which the Servers write their logs files too. Let's take a CHEAP $10 USB Flash Drive with a read/write limit of 500,000. If you do 100 read / writes a day it would be 13 years before you need to replace it. 1. A USB Flash Drive is $10 (and cheaper). 2. USB is going the way of a DoDo Bird. Your Server won't have a USB port in 13 years when you get around to replacing your USB Flash Drive. Thunderbolt (USB Replacement) is already out. Most new motherboards have a thunderbolt version. For example: ASRock Z87 Extreme4/TB4 or Gigbabyte GA-Z77X-UP5 TH Having said all that... You can run Linux fine from a Flash Drive and as long as you aren't running some database that is doing 1,000s of writes a minute, you will be onto new technology long before you wear out your $10 USB Flash Drive. While not getting it exactly, it seem hvm is needed for windows, to emulate bios and such You are running unRAID in PV Mode. Meaning, it doesn't have to emulate a full PC and all the various devices (which makes it slower). In PV mode, it has very minimal emulation and why you are getting close to "bare metal" speeds. You want to run Windows in PVHVM mode. It will emulate a PC but the Disk Drive, Network, Video, etc. are using Paravirtualized drivers. To Windows in PVHVM mode... You install Windows like you normally would in a VM and then load the GPLPV drivers. You will see SIGNIFICANT increases in Disk, Network, Video and slight improvements in memory. but the bzroot is a little less clear, I just assume this is the standalone OS portion of unRAID, so thats why... ramdisk for unraid, is unraid run in RAM? Maybe? The bzroot is Slackware Linux compressed. When you turn on your unRAID machine, it boots into the Linux Kernel (the bzimage) which then extracts the bzroot (Slackware Linux) into memory (which is read only). That is why you need plugins. It downloads and installs the various linux packages (into memory) each time you power on unRAID and it reads the config files for those plugins either on the flash drive or a cache drive. You could extract the bzroot and put it on the Flash Drive itself (there are few steps involved but fairly easy) and install those linux packages once. You could still have it read the config files from the Flash Drive or the cache drive. If you wanted to install Webmin for example... You go to Slackbuild site, download it, run their linux script and install it ONCE (Slackbuild Webmin). I also could just give you the package which you install and leave Slackbuild out of it. I could write a plugin that does all of that but that takes FOREVER considering I can do it one time in 5 minutes or less. If you wanted phpmyadmin, Mariadb, Flexget, etc... It's very easy to do. extra - reading the syntax guide this just means to take over the console session? I assume the reason this cannot be done with win8 is because win8 uses a full on GUI. Extra is just like what you would do booting up normal Linux via Grub (for example). You pass commands to the Linux Kernel. If you needed to load modules or tell it not to display the boot up or whatever. Windows 8 doesn't have a console. But you can access one if your VM is Linux, BSD, Solaris, etc. Quote Link to comment
smakovits Posted December 13, 2013 Share Posted December 13, 2013 testARCH /home # xl create win7.xml -c Parsing config from win7.xml xc: info: VIRTUAL MEMORY ARRANGEMENT: Loader: 0000000000100000->000000000019ec84 Modules: 0000000000000000->0000000000000000 TOTAL: 0000000000000000->000000007f800000 ENTRY ADDRESS: 0000000000100000 xc: info: PHYSICAL MEMORY ALLOCATION: 4KB PAGES: 0x0000000000000200 2MB PAGES: 0x00000000000003fb 1GB PAGES: 0x0000000000000000 Daemon running with PID 5454 xenconsole: Could not read tty from store: No such file or directory libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus: console child [0] exited with error status 2 when I do xl list, I see it listed but not sure if it is running. Try to connect with VNC, but it fails, any ideas what I might be doing wrong? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.