apandey

I'll try, though you should be able to look up terms like parity, RAID and JBOD on wider internet too In a RAID setup, all disks are used together, and every piece of data is spread across the drives uniformly. There is some redundancy by adding parity data to real data, which is also spread across drives. If you lose more than allowed drives, it is hard to recover anything because data is spread across drives at a very low level. Individual drives don't have identify able data themselves. Also, most raid setups will only allow expansion by adding drives of identical size and any extra space on bigger drives may not be used. The benefit is that since multiple drives work together, you can get better performance by spreading parallel IO across drives whenever possible Unraid is not RAID, but rather JBOD, where a bunch of disks with independent filesystems and data files are accessed via a virtual filesystem (shfs) on top of them. In theory, if you lose a drive, you only lose data on that drive, and other drives can still be independently read. Also, you can mix drives of any size together. For redundancy, you can protect the bunch of data drives (array) by adding dedicated parity drives (1 or 2) which keep a value that can be used to recompute the data on (1 or 2) broken drives if they fail. All drives are not used together, usually reads are from a single drive where data is, and writes are to a single drive + parity drives(s). The only time all drives are used together is if a disk fails and has to be rebuilt (data from all remaining drives + parity drives is needed to reconstruct a failed drive, just like RAID). Given above, the performance of unraid array is limited to a single disk speeds, but you have more flexibility in mixing drives of different sizes As for your hardware, it all depends on whether all components have drivers in unraid. You can test by booting unraid OS from a usb to see if all your hardware components are detected. Also, unraid works with hard drives at low level, so needs direct access to them. If your server has a raid controller, you need to be able to run it in pass through mode to avoid any surprises

If you could run it via cron, user scripts would do it for you. It has a cron trigger I don't understand why docker won't work. That is the preferred way to run things on unraid What is your use case for the tunnel? Maybe unraid already had a preferred way to do that

You need your home dns server to resolve that to an IP. And you need your clients (browsers) to use that dns server. For me, this is dns resolver in pfsense, resolving to haproxy frontend Further, you need something running at default web ports on the resolved IP address to proxy that host name to corresponding docker ip:port. For me this is haproxy running in pfsense. Without this layer, you can also directly resolve the name to docker IP, but then you will have to use name:port in browser

/config/go.txt is a startup script for unraid, if all you want is trigger something on every boot The user scripts plugin also allows you to schedule scripts at various triggers and events - like cron, array start / stop etc. Further, what you want to do might be possible via a docker container which can be set to auto start with array

I have haproxy running in pfsense which routes to specific containers wherever they are on the network Dns names will only resolve to IPs, if you want port translation, you need a reverse proxy or similar routing mechanism in between

The time isn't spent loading files from media, but rather communicating with docker service and enumerating out all the details for each container you are running The plugins tab is even slower, because it also seems to try and work out if any of the plugins have updates available

Unraid runs completely out of memory for the most part, the basic system including the UI are loaded from the image on usb into RAM during boot. There isn't much on USB that is accessed post boot. There is absolutely nothing on your shares and data drives that is used by unraid system System shares is used for docker images and VM config files. See https://wiki.unraid.net/UnRAID_6/Shares#Default_Shares Perhaps better to discuss what you are finding slow and try to diagnose why it may be so instead

https://wiki.unraid.net/Manual/Changing_The_Flash_Device

Yes, pulling where the connections are initiated from your end is a more secure setup than allowing the remote end to directly connect to you. There is a higher chance that someone can get physical control of remote end than the local one, and you don't want a way where they can connect back in directly Even better would be for the remote end to upload to a cloud service from where you can pull. That service then becomes a controlled interface with it's own security layer

Since you were using raid0 via a raid controller, there would be raid identifying tags and data on the drive which would offset the actual filesystem. This is why the drive isn't seen as an xfs formatted drive. I hope you did not corrupt it further trying to assume it's xfs and repairing it Unraid expects it's working with a passthrough controller with direct control of disks Thr safest option is to start with freshly formatted drives in new servers array and copy over the data even if it takes time. You can do it disk by disk slowly growing the target array, or all at once if you have backups to copy from

what you are planning will work, with a lot of reconfig / parity rebuild etc. You have absolutely no spare computer where you can connect those 4 drives, boot unraid, create desired array and copy between 2 servers? that will be simplest, and you can then drop in all 4 drives including parity into new server without having to rebuild parity The next simplest would be if you can connect all 4 new drives to your current server, then passthrough them into an unraid VM, then do the same process.

can you describe what paths you have mounted to krusader and which paths are you moving across? mover will not move any open files. you can use open files plugin to verify you can run a btrfs scrub to check. What path are you giving to the VM?

how are you doing this? best if you can share diagnostics bad connections / bad cables are more prevalent than bad disks. check those first

Yes, assuming no data has changed on the remaining valid data drives. Parity itself is not enough to rebuild a drive, it needs parity + data drives. The data drives should be unchanged to match what was on offlined parity drive If the goal is to have dual parity, then yes, that's fine. I am assuming there are separate backups for any important data anyway

Most usb flash drives have a universally unique hardware level identifier. Unraid ties this UUID to your license. This mechanism is not available on SSD / HDDs What you need to know is * some usb drives don't have a unique id, in which case they may show up as not compatible (no id) or blacklisted (same id across drives) * in theory, it's possible to use a usb to another interface (ssd, SD etc) where the usb controller provides a unique id. In this case license will be ties to the usb adapter device. But there are practically no such devices with unique ids. Most just use some generic id's across devices

If they are all SFP+ interfaces, you can connect your pfsense + 7 other 10Gig devices to your aggregation switch. All connections are negotiated point to point. So that meets your objective already

I have a 4u case, which came with 4 Norco style high airflow 80mm fans on the fanwall. It was noisy and not cool enough I removed the fanwall frame completely and replaced it with 3x 140mm noctua fans. 140x3 forms a 420mm fanwall that perfectly fills up the space where my fanwall was. The setup is cooler and relatively quiet. I got about 10C improvement in drive temps

Great. Use case is clear 🙂 I am also a novice at zfs right now, but have plenty of experience with other raid / raid like systems. So take my advice with that caveat in mind I guess you are aware at how zfs uses vdevs. You might want to think that through, because within a vdev, there would still be limits on performance for a multi tenent use case like yours. I am myself trying to understand the best practices on zfs topology, and as of now, unraid doesn't allow all config via UI For sequential IO, you can get close to theoretical combined capacity of all disks, which should be in quite some contrast to unraid array. But random IO across multiple users still has the problem of all participating disks having to seek to same location every time, bringing performance closer to single disk Good Luck, and hope you will get more experienced response than me

Correct. Data is distributed across drives in zfs raidz, so all disks in a vdev are at play together

It's not clear to me what that situation is. I asked about your objectives for that reason If there was a single way to "correctly" set up a network, other ways won't exist. If you are looking for best practices, you need to clearly put out what you want. How many devices, how you want them to communicate, what you want out of it (speed / security / reliability / budget / infrastructure) And then, if you get multiple ways to do it, you have to accept there are multiple correct ways and start to learn the pros and cons of why those choices are being made Right now I don't even know whether you are talking about OSI layers or your device connection topology when you mention layers. That's not where I would start, I would rather ask what makes layers relevant to this discussion at this point?

Echo points made by @itimpi, sounds like you need more/bigger caches and use them selectively An additional suggestion - maybe you can also look at mover tuning plugin and schedule it aggressively with disk free threshold settings

You seem to be interchangeably using pool to describe both unraid array and unraid pools. To add to the confusion, zfs also shares the pool terminology to describe it's own construct Unraid needs an array, but an array is just a JBOD protected by parity drives. You don't seem to want that. But it can be done with a single drive, even usb. So I suggest you attach a smaller spare drive or a separate usb drive for array. To run a zfs pool, you have to create it as an unraid pool (if running 6.12 rc) or using zfs plugin (if running 6.11). The process to use them is slightly different though, so I will explain the simpler 6.12 method which is what it will be going forward If all you want to use is zfs pool, you can expose it as a cache only share pointing to the corresponding pool For your use case though, I have to ask why use unraid at all? Something like truenas is more suited to a pure zfs use case in today's landscape. Do you plan to use other unraid features?

What version of unraid? The plugin fix isn't working? Maybe post diagnostics

What specific capabilities are you after? There is no best network, only one that works or doesn't work against specific goals

I am not sure you can plug in a new flash drive into a running system and make it work - I am assuming you are doing something like that If you are worried about preserving config, the best bet is to try backing up the config (from running system + screenshots etc). But in the end, to go to a new USB, I think you will need to reboot If I am wrong, would be happy to be corrected and learn