As I've had many issues in the past with disk and file system handling, I'm reaching out to you to build a new system. The hardware is already set:

- 1 Gbit/s LAN (upgrading to 10 Gbit/s later)
- 128 GB RAM
- AMD Ryzen 9 3900 (12 cores)
- 2 x 2 TB NVMe (for cache)
- 8 x 22 TB SATA 6Gb/s ( 7 + 1 spare)

 

Now, to my main question: which disk file system would you recommend? I have 10 active users, and the server will be used for various tasks:
- Sharing and syncing files
- Running a movie server
- Hosting some VMs (on the cache drives)
- Hosting a few websites

- VPN

 

In the past, I ran into issues with ZFS expansion not working as I expected. With this setup, I’m aiming for a different approach. As this configuration provides plenty of storage for my needs, I’m not planning to expand it; this is more of a "closed system." If I need more storage, I’ll rebuild the setup entirely. (So flexibility with storage expansion isn’t a major requirement here.)

The only planned upgrade is the LAN speed, from 1 Gbit/s to 10 Gbit/s.

For file sharing, I’ll be using apps rather than traditional shares, so share creation isn’t a big concern. I plan to set up a few main shares:
- Main data (movies, personal files for sharing and syncing)
- VM data
- Container data
- Config backup

 

This will be the main folder structure:

```
data
├── download
│   ├── incomplete
│   └── complete
│       ├── books
│       ├── movies
│       ├── music
│       └── tv
└── media
│   ├── books
│   ├── movies
│   ├── music
│   └── tv
└── personal
    └── ...
```

 

The plan is to go with a ZFS pool, allowing the 10 users to read and write across multiple disks simultaneously, which should boost performance during simultaneous read/write operations.

With my last setup, moves between datasets (essentially shares in Unraid) were extremely slow. However, I’ve set up the folder structure above to avoid that issue this time.

So... what would you recommend? What are the pros and cons of this solution regarding disk and filesystem layout?

I would recommend using unraid 7 beta 2 and setup a non array system with 2 pool devices.
a raidz2 and mirror

2 x 2 TB NVMe (for cache) in a zfs Mirror or btrfs mirror 

Have all  8 x 22 TB SATA 6Gb/s ( 7 + 1 spare)
put all 8 into a zfs raidz2 (you can lose 2 disks and not lose any data.)
Put all data, applications, vm, data, etc on this pool have a rsync copy / plugin appdata to copy to the 2x2TB cache pool for a backup...

zfs expansion only works well when with degrading a smaller disk and putting bigger disk sze if you put a 2tb replacement in a 4 the whole zfs will drop space to compensate you would need to match 4 across the board and won't see space improve until all drive have the same capacity...

Iv'e been happy with this setup for a long time:




You need a min of 4 disk to use raidz2 in theory you could have 2x pool of 4x 22TB in a raidz 2 and zfs clone and copy between them
These are just example of many things you can potential do. But as you said you had issues with ZFS. It how you want to interact with them and what you want them to do.

I run a plex server... with 3x 10 TB in a raid z1 i have ran this on unraid for a good 10 years..

There are quite a bit of random topics on how i should procedure and what would i run. Please see a recent forum post of a similar nature regarding best practices:

While Vm can take a performance hit as they should be on the faster drives. I can see you using 3 pools
as another example
2x nvme xfs array
no cache
2 pools of 4x22TB as zfs raidz2
You could in theory then use the array for vms, dockers etc...
Ideally for easy of management i would recommend staying with zfs and a pool only setup.

as for your data folder hiarcy here is an example that i'm running on unraid v7 beta 4:


example of some use scrpts i run:

Lets try to answer some questions...
-Sharing and syncing files:
Userscript and rsync command is my go too... chat gpt can help here sometimes... I will glady assist where I can. As will other here on the forum... There are quite a few posts here and there that can cover this form user script to dockers like sync thing... some Linux shell script will be done here...

- Running a movie server

It Jellyfin or plex usually for theses sometimes kodi... I prefer Plex... this is a docker that can be ran...
My current goto is plex as movie server. My current Plex has over 6 TB in space and I have a good complete collection of TV Series and Movies... alot of disk rips from makemkv. 

- Hosting some VMs (on the cache drives)
unraid uses libvirt KVM using qemu software. It would be best for performance to run this on the fastest drives whether it be the cache or array on how you set up your 2x nvme.

array disk 1 and parity / cahce mirror btrfs or zfs mirror...

- Hosting a few websites
* for security I would recommend this being a lxc / VM one to be able to install 3rdpaty libs... While dockers exist it would be easier to assign a mac / ip and install nginx / appache as web server and snapshot, backup to clone to duplicate and make similar websites...

latter you could run docker nginx as a reverse proxy... Ask on the forum for support...

- VPN

in terms of vpn, I would advise you to be smart and secure. There are quite a few dockers out there and services. Lately. Unraid partnered with Tailscale. It is fairly easy to set up an account and connect multiple devices.

Please note tailscale free can only have 3 users and ?x amount of devices. not all VPN are free..
ATM i'm use twingate similar to what tailscale did but found it faster and easier to set up a docker that exists to add to unraid as example...
 

Edited October 30, 20241 yr by bmartino1
typo