Leaderboard

A clear disk also has no impact on parity2. Same as parity1. All zeros doesn't affect the parity2 result either.

When you did this, you eliminated the preclear status. I.e., a precleared disk has only zeros written to every byte on the data area of the disk except for a special precleared signature. Formatting a disk writes a lot of data to a disk. Let's realize what happens when you add a new disk (straight out of the packing) to Unraid. It first writes zeros to every byte on the disk. (This is Unraid's clearing process.) This assures that Parity1 is valid. (At this point, I am not sure how Parity2 become valid.) It now adds the disk to the array and the disk must next be formatted which adds a fair amount of data to the disk to built the structure of the file system. As it writes to the formatting information to the disk, parity is updated for each and every byte written to that disk as it will be for every byte written thereafter. What happens is that if you take a disk straight from the preclear process and add it to the array, it looks for and finds the preclear signature. It will now skip its clearing process, adds the disk to the array and begins the formatting process, writing the parity information as it does so. Why a preclear script? Well, way back in olden times, Unraid took the entire array off-line while it did the clearing process. This wasn't long for an 80GB HD but as hard drives got larger, it became a real issue. Hence, the first preclear script appeared back in 2008. But the script did much more than just clear the disk. It also tested every byte on the disk. Back then, hard disks were not as reliable as they are today. Early failures were more of a problem. Folks started wanting to extend testing for 50 to 100 hours which would catch most disks that were going to fail early in their lives. (I can remember back in the 1990's every computer manufacturer burned in their systems for 72 hours before shipping to catch this type of failure!) The preclear script was perfect for this type of burn-in and testing. The preclear element of the preclear scripts is now a non-issue (as the array is not taken down while it runs) but the testing part is still with us.

It is possible to setup a LAN to LAN VPN connection between two Unraid systems running Wireguard. The steps below should work on simple networks, if yours is more complicated then you'll need to figure out how to adapt it. This assumes you already have at least one working WireGuard connection and are familiar with how it works. First, gather the following information for your two networks. The names and numbers below are samples, you'll need to adjust for your situation: Network1: 192.168.1.0/24 Router1: 192.168.1.1 Unraid1: 192.168.1.50 Endpoint1 DNS: network1.duckdns.org Endpoint1 Port: 52001 Network2: 192.168.2.0/24 Router2: 192.168.2.1 Unraid2: 192.168.2.50 Endpoint2 DNS: network2.duckdns.org Endpoint2 Port: 52002 VPN Tunnel: 10.252.100.0/24 Unraid1 tunnel IP: 10.252.100.1 Unraid2 tunnel IP: 10.252.100.2 A few things: Note that Network1 and Network2 *have* to be different. For instance, you can't connect two networks that both use the 192.168.0.0/24 subnet. You also need to pick a VPN Tunnel subnet that is not being used on either network, it must be unique. You need to setup DDNS for both networks as well. LSIO has a nice Duck DNS docker you can use if needed. On Unraid1: Create a new Tunnel named "Network1-Network2". Don't add a peer to an existing tunnel, it is better if this is a separate tunnel so you can easily turn it off without affecting your other WireGuard connections. Switch to Advanced Mode Click "Generate Keypair". These are the private and public keys for Unraid1, you will need them later. Set the "local tunnel network pool" to the "VPN Tunnel" you chose above Set the "local tunnel address" to the "Unraid1 tunnel IP" from above Set the "Local endpoint" to the "Endpoint1 DNS" and "Endpoint1 Port" defined above Click Apply Click Add Peer Name it "Unraid2" Choose "LAN to LAN access" Click "Generate Keypair". These are the private and public keys for Unraid2, you will need them later. Click "Generate Key". This is the preshared key, you will need it later. Set the "Peer tunnel address" to the "Unraid2 tunnel IP" defined above Set the "Peer endpoint" to the "Endpoint2 DNS" and "Endpoint2 Port" defined above Set "Peer allowed IPs" to the "VPN Tunnel" and "Network2" defined above, with a comma between. i.e. "10.252.100.0/24, 192.168.2.0/24" Click Apply Start the Tunnel On Router1: If UPnP is disabled, setup a port forward for "Endpoint1 Port" (UDP) that points to "Unraid1" Setup a static route for all of "Network2" that is routed through "Unraid1". Note that if your router asks for a subnetmask, 255.255.255.0 is the equivalent of /24. Setup a second static route for "VPN Tunnel" that also is routed through "Unraid1". On Unraid2: Create a new Tunnel named "Network2-Network1". Switch to Advanced Mode Copy the private and public keys for Unraid2 that you determined above. Take care to use the correct keys or none of this will work. Set the "local tunnel network pool" to the "VPN Tunnel" you chose above Set the "local tunnel address" to the "Unraid2 tunnel IP" from above. Set the "Local endpoint" to the "Endpoint2 DNS" and "Endpoint2 Port" defined above Click Apply Click Add Peer Name it "Unraid1" Choose "LAN to LAN access" Copy the private and public keys for Unraid1 that you determined above Copy the preshared key from above Set the "Peer tunnel address" to the "Unraid1 tunnel IP" defined above Set the "Peer endpoint" to the "Endpoint1 DNS" and "Endpoint1 Port" defined above Set "Peer allowed IPs" to the "VPN Tunnel" and "Network1" defined above, with a comma between. i.e. "10.252.100.0/24, 192.168.1.0/24" Click Apply Start the Tunnel On Router2: If UPnP is disabled, setup a port forward for "Endpoint2 Port" (UDP) that points to "Unraid2" Setup a static route for all of "Network1" that is routed through "Unraid2". Setup a second static route for "VPN Tunnel" that also is routed through "Unraid2". Hopefully at this point your tunnels will connect and devices on one network will be able to reach devices on the other network (by IP address at least, probably not by name) Troubleshooting this will be tough, there is a lot of room for error. I don't have a lot of advice here, just double check that you are using the right values for Unraid1 vs Unraid2 and Network1 vs Network2, etc.

tldr: If you are running Unraid OS 6 version 6.8.1 or later, the following does not apply (mitigations are in place). If you are running any earlier Unraid OS 6 release, i.e., 6.8.0 and earlier, please read on. On Jan 5, 2020 we were informed by a representative from sysdream.com of security vulnerabilities they discovered in Unraid OS. Their report is attached to this post. At the time, version 6.8.0 was the stable release. The most serious issue concerns version 6.8.0. Here they discovered a way to bypass our forms-based authentication and look at the contents of various webGUI pages (that is, without having to log in first). Then using another exploit, they were further able to demonstrate the ability to inject "arbitrary code execution". Someone clever enough could use this latter exploit to execute arbitrary code on a server. (That person would have to have access to the same LAN as the server, or know the IP address:port of the server if accessible via the Internet.) Even in versions prior to 6.8.0, the "arbitrary code execution" vulnerability exists if an attacker can get you to visit a webpage using a browser that is already logged into an Unraid server (and they know or can guess the host name of the server). In this case, clicking the link could cause injection of code to the server. This is similar to the CSRF vulnerability we fixed a few years ago. In summary, sysdream.com recognizes 3 vulnerabilities: That it's possible to bypass username/password authentication and access pages directly in v6.8.0. That once authentication is bypassed, it's possible to inject and have server execute arbitrary code. That even if bug #1 is fixed, #2 is still possible if attacker can get you to click a link using browser already authenticated to your Unraid server (6.8.0 and all earlier versions of Unraid 6). Mitigations are as follows: First, if you are running version 6.8.0, either upgrade to latest stable release, or downgrade to an earlier release and install the sysdream mitigation plugin. We are not going to provide a mitigation plugin for 6.8.0. If you are running any 6.6 or 6.7 Unraid release, the best course of action is to upgrade to the latest stable release; otherwise, please install this mitigation plugin: https://raw.githubusercontent.com/limetech/sysdream/master/sysdream.plg This plugin will make a small patch to the webGUI template.php file in order to prevent arbitrary code execution. This plugin will work with all 6.6.x and 6.7.x releases and should also be available via Community Apps within a couple hours. We are not going to provide a mitigation for Unraid releases 6.5.x and earlier. If you are running an earlier release and cannot upgrade for some reason, please send us an email: [email protected]. I want to thank sysdream.com for bringing this to our attention, @eschultz for initial testing and fixes, and @bonienl for creation of the sysdream mitigation plugin. I also want to remind everyone: please set a strong root password, and carefully consider the implications and security measures necessary if your server is accessible via the Internet. Finally, try and keep your server up-to-date. VULNERABILITY_DISCLOSURE.pdf

It appears the dam has finally burst! These processors have been hard to find in retail channels and when available were expensive and in limited quantities. As of today, Provantage has 58 Intel Xeon E-2278G processors in stock. https://www.provantage.com/intel-cm8068404225303~7ITEP6KU.htm

Yes, 3900X is cheaper and has more cores. I was actually considering that option; however, I am buying the Intel because it has a built-in iGPU that works for Plex/Handbrake transcoding. The AMD option means I need a discrete GPU and the Nvidia unRAID plugin/special unRAID build. I ended up with the E-2288G which has a 3.7 GHz base clock which is comparable to the 3900X although that is not a huge concern to me. I would go AMD in a heartbeat if they had decent APUs with something more than 4-core processors and if they were fully supported by Plex (some claim they can get AMD iGPUs working in Plex). There are also more server motherboards for Intel Xeon that support IPMI. I have found only one Ryzen MB with IPMI (ASRock Rack X470D4U). This is likely my last Intel CPU purchase. By the time I am ready for another upgrade, hopefully, the issues holding me back from going AMD this time will be moot.

That because those SMR drives don't read the platters when they know there's no data there, do a full write and then you'll get the actual read speed from the platters.

If you have a large data from a user using sql lite could be very bad. I lost my first Nextcloud install because I choose to do the sql lite I was transferring so much data that the dB just quit and could not recover. Move to mariadb (MySQL essentially) haven’t had a problem. Just my input. Sent from my iPhone using Tapatalk

Remember when I warned about dumb questions? 😉 Yes, that's exactly what I did, and you described the issue/resolution perfectly. Thanks for that! All working perfectly again Cheers, Shaun

Afraid that's not how this works. How did you upgrade to 6.8.2? If you went via an Unraid notification (or system tab), then you have uninstalled the DVB drivers. Go to the plugin, select 6.8.2 and install it. If you did upgrade via the plugin's page then this version doesn't look compatible with your setup and more investigation is needed. For a brief but maybe confusing explanation - this plugin does not install DVB drivers. It gives you access to custom versions of Unraid with the drivers added. Upgrading to vanilla Unraid (via a notification/system tab) will revert back to regular Unraid and remove those drivers. Therefore, the ONLY way to retain drivers and upgrade Unraid is to download and install 6.8.2 via the DVB plugin page. Mistake that a lot of us have made (myself included) - install plugin -> install custom Unraid -> Unraid is upgraded -> upgrade via normal route -> wonder where drivers have gone. If this isn't what you've done, apologies.

Hello from germany, I have some issues with USB passthrough when using USB sticks, USB HDD and USB Bluetooth devices on my VMs. Mouse and keyboard seem to work just fine with some minor lag. When either the devices or USB controller are being passed through (OS doesn't matter if Linux of Windows) to the VM they are being properly recognized and installed. Unfortunately, when I'm trying to access those on VM installed USB devices, for example when I'm trying to copy something, the data transfer is getting slower and slower until the USB-Device completely disappears. After that USB device is no longer visible in Windows hardware manager too. Another example could be a headset, it's being recognized and installed too, but when I'm trying to use it the sound is delayed, or it stutters. Mice and Keyboards seems to work but when the host is overloaded, they stutter and react very sluggish/slow. When I'm on UNRAID-Host those problem doesn't appear, the Datatransfer is normal. The IOMMU groups are separated for example: IOMMU group 12: [1b21:2142] 01:00.0 USB controller: ASMedia Technology Inc. ASM2142 USB 3.1 Host Controller IOMMU group 13: [1b21:2142] 02:00.0 USB controller: ASMedia Technology Inc. ASM2142 USB 3.1 Host Controller What have I already tried: 1. New configuration of UNRAID 2. Different VMs and different OS (Windows, Linux) 3. Dedicated USB PCIe card 4. USB controller with vfio-pci.ids= impossible 5. Different USB controllers 6. USB hub with its own power supply My specs: Model: i9 7900x 32GB M/B: Micro-Star International Co., Ltd X299 TOMAHAWK (MS-7B05) Version 1.0 BIOS: American Megatrends Inc. Version 1.90. Dated: 07/31/2018 CPU: Intel® Core™ i9-7900X CPU @ 3.30GHz HVM: Enabled IOMMU: Enabled Cache: 0 KiB, 0 KiB, 0 KiB Memory: 64 GiB DDR4 (max. installable capacity 3072 GiB) GPU-Passthrough work without any issues, I have tried RTX 2080 or GTX 970, and other non-USB devices like TV-Cards or NIC-Cards work just fine. I've exhausted all ideas I could came up with, that’s why I need your help. Many thanks in advance.

You can also press the key 'F1' to toggle help on or off.

Are you referring to the GUI’s built-in help? If so this can be toggled on/off at the page level by clicking on the ‘?’ Symbol on the top right of the GUI menu bar. It is also typically accessible at the field level by clicking on the fields description.

A number of us are having the same problem and no one has responded with any idea of the reason or a solution. Both containers can connect to each other. proxy is working for both internal and external connections to both containers... documentserver.domain.com "refused to connect"

This post is exactly why I trust the unraid team, great job in the transparency !!!!

is hugepages related to thata hugebitch...... I guess I missed that Linus video.

Can't really answer that question. It's not about quantity as much as its about use-case and applications. I've never needed hugepages for any of my VMs because I'm not running databases, video encoding, etc. The benefits of hugepages are pretty app-specific. Linus saw the benefit specifically in a video he did where it featured a lot of video encoding being done and there was a pretty dramatic impact to overall performance as a result. Then your VM will give you an error upon trying to start it. What is important to remember is that to change your allocation for hugepages, you need to adjust the lines in SysLinux and reboot. There are technically methods that allow you to do this while the machine is running, but I highly advise against that as you may not get contiguous memory allocation then.

Hugepages support is built into Unraid natively, but to enable it, you must do two things: 1) Navigate to Main > Flash Device Settings (click the Flash on this page). After "append" and on the same line, add the following: hugepagesz=2M hugepages=16128 hugepagesz=2M hugepages=X Change X to represent the # of hugepages you want to allocate. Each page is 2 MB based on the first variable. So for example, if you wanted 16GB of RAM, you'd use the following: hugepagesz=2M hugepages=8064 You obviously will need to increase this for the # of VMs you wish to use this with. Here's what mine looks like as an example: 2) Edit the your VM and switch to form-based edit mode (the toggle switch in the top right on the Edit VM screen). Add <hugepages/> to the <memoryBacking> section like so: Save your VM, reboot your server and fire it up.

That is why many have requested support for multiple arrays within a single system.

Here's something to test. Please disable the use of your AMD GPU in the VM and use VNC. Then keep the USB controller passed through. See if that works. If so, this is a GPU, not a USB controller issue. The reason this test is a good one to perform is simply because AMD-based GPUs are notorious for causing the exact issues being described here (VM works fine until shutdown/restart). This is because most AMD GPUs don't support function level resets which are vital for good experience in a VM. The same could be the case with the USB controller, but that's harder to say as we don't have as much experience with those devices. In general when it comes to any VFIO / PCI device assignment, some hardware just plain doesn't work well with it. The VFIO project aims to do the best possible job it can to support generic PCI device assignment to VMs, but there are just some cases where the way the hardware was designed, it just doesn't work correctly and there is little we can do here at LT to resolve these types of issues. That said, if things were working for you on a previous release that aren't working now, please be sure to include that detail in your posts and be sure to mention which version of Unraid was the last known working version for your setup.

I think you should be able to use "auth_basic off;" for the location you want to exclude.

Something else I wanted to add, as long as we're talking about security measures in the pipe: we are looking at integrating various 2-Factor solutions directly in Unraid OS, such as google authenticator.

I haven't "danced" around anything, sorry if it appears like that. How does this apply in an Unraid server environment? Yes this is something we're looking at. why? why? There is only one user: root You can set file permissions however you want using standard linux command line tools. Again, what are you trying to accomplish? We do have plans to introduce the idea of multiple admin users with various roles they can take on within the Management Utility. For example, maybe you create a user named "Larry" who only has access to the Shares page with ability to browse shares only they have access to. However this functionality is not high on the list of features we want/need to implement. Earlier you were confused by my term "appliance". What this means is the server has a single user that can manage the box. If you don't have the root user password, all you can do is access shares on the network that you have permission for, and access Docker webUI's - but most of these have their own login mechanism. Things like the flash share exported by default, new shares public by default, telnet enabled by default, SMBv1 enabled by default, etc. are all simplifications to reduce frustration by new users. Nothing more frustrating that creating a share and then getting "You do not have permission..." when trying to browse your new share. We are trying to reduce the swearing and kicking of dogs by new users just trying to use the server. Eventually everyone needs to be more security conscious - and in that spirit we are working on "wizards" that will guide a user to setting up the correct settings for their needs. I hope this starts to answer some questions and sorry if I came across flippant to your concerns, but trust me, security is a foremost concern and to have someone imply otherwise ticks me off to be honest.

This is a load of B.S. While I appreciate the sentiment of your post (wanting to improve security), it is not helpful to simply complain. What is helpful is to point out specific attack vectors that we can address. Unraid is rapidly evolving from a simple NAS mainly used by tech-savvy home users to a more general platform with a wider range of users. It used to be the introduction of some bug that causes customer data loss that kept me up a night. These days, having a bug that presents a security risk is far more worrisome. So don't tell me we don't take security seriously. That said, there is a trade-off between making the server easily accessible for a first-time user vs. locking it down so tight no one can figure out how to even get in. I'll give you an example. By default we export the 'flash' share as a public share. Some people's hair catches on fire because of this. But the reason it's done this way is that after a user creates a bootable USB flash a very simple test is to see of the 'flash' share shows up in network explorer. There are other reasons it's handy to have this public for at least some amount of time. These days we have an icon next to the flash share if it's public, where rollover warns about this. Moving forward we are developing an initial configuration wizard that will guide a user in setting up the level of security appropriate for them.

Hey, did you find a possibility to automate to power down the graphics card after shutting down the VM? Greetings, ViproXX

We have multiple-pool support coming in 6.9 release. After that we can look at adding a UI to create vdisks that map to iscsi targets. Please do me a favor however: please move any non-Unraid discussion to another forum.

StarTech.com 4 Port PCI Express (PCIe) SuperSpeed USB 3.0 Card Adapter w/ 2 Dedicated 5Gbps Channels Chipset: Renesas/NEC - µPD720202 Findings: Purchased this because it had 4 individual USB controllers on the board thinking the extra price would allow for more flexibility with my VMs. Requires SATA/Molex power, works fine when booting native to Windows 10. It is recognized in Unraid under Settings -> Hardware however when passing through the PCIE card to a VM, while the card is detected, it always has an error with Code=10. This is with the standard Windows drivers and also with the drivers from the manufacturer's website. There are other posts online with the same result (Code=10 error in Device Manager) but not promising workarounds. I suspect this is something at the Linux level as when I do not pass through the PCIE card and hook up USB devices directly to the card, they also are not recognized by Unraid. I give up on this card.

USB Card roundup for Windows 10 and High Sierra Virtual Machine Use 1st Card - AUKEY 4 Ports PCI-E To USB 3.0 Expansion Card Chipset: VL805 Findings: Got an error in my syslog during boot and card was unable to be used either natively in Linux or passed through to Windows or OSX VM 2nd Card - Anker Uspeed PCI-E to USB 3.0 2-Port Express Card Chipset: VL805 Findings: Got an error in my syslog during boot and card was unable to be used either natively in Linux or passed through to Windows or OSX VM 3rd Card - Sienoc USB3.0 7Port PCI Express Card Chipset: - Renesas D720201 Findings: Worked natively in Linux or passed through to Windows, but not detected by OSX VM 4th Card - Inateck KTU3FR-5O2I Chipset: Fresco FL1100 Findings: Worked natively in Linux, and worked beautifully in Windows or OSX VMs, able to use my hardware KVM switch with either, but unable to power my server off with the card plugged in, as soon as it powered off, it powered on again. @aptalca has had the same issue with a Fresco FL1100 card and we both have Supermicro motherboards. 5th Card - ORICO USB3.0 4 Port PCI Express to USB3.0 Host Controller Card (PME-4U) Chipset: Fresco FL1100 Findings: Worked natively in Linux, and worked beautifully in Windows or OSX VMs, able to use my hardware KVM switch with either, but unable to power my server off with the card plugged in, as soon as it powered off, it powered on again. @aptalca has had the same issue with a Fresco FL1100 card and we both have Supermicro motherboards. I ordered this one to try and exclude the possibility it was an inherent Inateck fault, but after testing this one, I think we can conclude there is an issue with the FL1100 chipset and Supermicro boards. 6th Card - KALEA-INFORMATIQUE Controller Card PCI Express (PCI-E) to 3.1 USB Chipset: Asmedia ASM1142 Findings: Success!! Works in Windows 10, works in OSX natively both, without installing extra drivers, allows my server to powerdown and stay down. Other possibilities: Well I'm running out of chipsets to try, only thing I could consider is an expensive Sonnet card which uses the Fresco FL1100 chipset but had four controllers on the card which can in theory be split amongst VMs so one controller to each VM. I don't hold out much hope this will actually work well given it's got four of the damned chips that won't let me powerdown in the first place. One other thing I did try was using the Renesas based chipset to boot from and passed through my native USB ports, that didn't work well. Not sure why, but I might investigate that a bit more if I have no other options. Conclusion If youi've got a Supermicro motherboard, then Asmedia ASM1142 based USB cards are the way to go, for any other motherboard, then either the Asmedia ASM1142 chipset or the more common Fresco Logic FL1100 based cards would be fine, but if you notice that you have issues with powering down, the card may well be the culprit and try an Asmedia card instead. It's been..... emotional

I had this issue as well recently, I found the only way to get rid of them was to go the actual disk where the files reside and delete them from there, so /mnt/diskx/Films/ instead of /mnt/user/Films/. When I did this I was able to delete the files successfully. Good luck