ZFS plugin for unRAID


steini84

Recommended Posts

Hello!

 

Plex metadata and stuff were on cache only.

Plex libraries were on unraid array and that's where I had perf issues, I barely could direct play 1 4k movie, and 4k movies with high bitrate were always buffering that's why I moved to Xpenology that works as expected EXCEPT that HDD hibernation doesn't work so HDD always spin and power consumption is high :(

But that was like 2 years ago, I don't know if Unraid Arrays are better now!

Edited by Valiran
Link to comment
52 minutes ago, Valiran said:

Hello!

 

Plex metadata and stuff were on cache only.

Plex libraries were on unraid array and that's where I had perf issues, I barely could direct play 1 4k movie, and 4k movies with high bitrate were always buffering that's why I moved to Xpenology that works as expected EXCEPT that HDD hibernation doesn't work so HDD always spin and power consumption is high :(

But that was like 2 years ago, I don't know if Unraid Arrays are better now!

 

A high bitrate 4K file is likely to be around 5-10MB/sec, I doubt this was an issue with array performance.   Plex has always struggled with 4K where transcoding is required.  Worth another try, things may have improved.  

Link to comment

It's possible you had one very slow disk caused by bad disk or bad cable or similar if there were those kinds of performance issues on the unraid array - if I recall directly it will only go as fast as the slowest drive.  It should be enough for playing video media - you really only notice it when you want to copy large amounts of data to or from it.

 

Personally unless you have a need to use a large amount of differing sized disks I wouldn't bother with the unraid array when you are already experienced with a much better tech - ZFS (which is purportedly going to have a much tighter integration in the next version). ZFS will add some really core benefits like actually tell you when things corrupt and offer to repair them - the unraid array will only rebuild a disk really - it doesn't do much else than that.  And if it's speed you want - the capabilities it has for SSD mirrors are also pretty awesome.

Link to comment

Yes I know ZFS is cool, but I can't manage to find a guide for the dumb on ZFS.

For example, I have 6 hdd (5*12tb + 1*18tb).

On those disks I have dumb things that I don't care to loose BUT thoose dumb things are like 2Tb so I can't dedicate a HDD to that.

I have my personnal photos and my plex librairies. Personnal Photos don't need to be fast, Plex Librairies need to be fast, both need to be protected by a raid system like raid5 (or raidz1?).

And from there, I don't know how to create my pools and everything else :)

That's why I want to stick to a simple thing !

Link to comment

First thing that comes to my mind, according to the required space, set a raidz or raidz2 and dedicate your 18Tb drive as a spare.

https://docs.oracle.com/cd/E53394_01/html/E54801/gpegp.html

https://docs.oracle.com/cd/E19253-01/820-2315/gcvcw/index.html (french one)

 

Proceed to a safe test; with no data in the array; to check that in case of a drive failure, the 18Tb drive is automatically mounted. Just remove the SATA cable to simulate a drive failure.

 

When creating the raidz, set the autoreplace property.

 

Quote

 

Edited by gyto6
  • Like 1
Link to comment
4 minutes ago, Valiran said:

What you've linked if about command lines. The ZFS addons for Unraid are not graphical friendly? 

 

Nope. If Unraid add ZFS support, it should be a bit more graphical. But with many caveats probably as they are tons of properties to customize a Zpool and a dataset.

Edited by gyto6
Link to comment
2 minutes ago, Valiran said:

So if I have to go ZFS, I better use TrueNAS, at least until full support is added to Unraid

 

No need to come back to Unraid if you're only planning to use Plex and to store personal data on your server I think.

Edited by gyto6
Link to comment
1 hour ago, Valiran said:

So if I have to go ZFS, I better use TrueNAS, at least until full support is added to Unraid

Truenas is probably more complicated than using the zfs command line. I wouldnt go there you’d be disappointed. Also getting a basic zfs system running on y raid is pretty simple really and there are a lot of friendly people on here that would help you. However if you’re the kind of guy that isn’t really into technology or willing to spend a bit of time learning then I might agree wait for official zfs support. Or just get a Mac and put plex on it and use iPhoto. 

Link to comment
15 minutes ago, Valiran said:

Yes if you have something to share, I may have something to read :D Thanks !

First, you'll find many informations on Aaron Toponce website : https://pthree.org/2012/04/17/install-zfs-on-debian-gnulinux/

 

Second : In Situ scenari, I wrote this tutorial to optimize ZFS for another application. I tried my best to explain ZFS so it might worth the shot. https://github.com/photoprism/photoprism-contrib/tree/main/tutorials/zfs

 

Third : ZFS/Unraid Compendium from @BVD https://github.com/teambvd/UnRAID-Performance-Compendium/blob/main/README.md

 

Quote

ZFS, while powerful, isn't for everyone - you'll need not only have the desire/drive to do some of your own background research and learning, but just as importantly, the TIME to invest in that learning. It's almost comically easy to have a much worse experience with zfs than alternatives by simply copy/pasting something seen online, while not fully understanding the implications of what that exact command syntax was meant to do.

 

Edited by gyto6
  • Like 1
Link to comment

OK have sent you the cheat sheet, which is sort of my raw notes to refer to when I want to look something up.  All of those things have been part of my journey.  I wish someone had sent me that in the beginning.

 

That will give you a starting point upon which you can take and discover from.

 

And while some say ZFS isn't for everyone, as far as I know, on unraid it is the only alternative to the unraid array.  I don't think mdadm works though it's probably possible.

 

It's also the only viable self healing file system that I'm aware of and it's worth its weight in gold.  I had an overheating SAS card once (though I didn't know it).  I was getting errors on 5 of the 6 disks in a Raidz1.  I couldn't believe how ZFS managed to keep all my data safe through the months it took me to figure that one out.  And now I have Raidz2 :D

 

ps. there is a lot of information there, you don't need most of it. so please don't be overwhelmed.  Probably just focus on zpool create and zfs create to begin with.  And learn what the options I've listed for those two commands mean. :D

Edited by Marshalleq
Link to comment
18 hours ago, Sally-san said:

I see that "expand" has been added to ZFS for raidz configs, is this feature part of the ZFS plugin yet? Or do we have to wait for and update?

 

If it is part of the Unraid ZFS plugin already, how do we use it. I'm still a newb.

ZFS "plugin" downloads the last ZFS version from OpenZFS github, so the functionality is there.

image.png.ae8efda9ff033bc03080f6d242017d90.png

Edited by gyto6
Link to comment
On 10/27/2022 at 3:58 AM, Sally-san said:

If it is part of the Unraid ZFS plugin already, how do we use it. I'm still a newb.

9 hours ago, gyto6 said:

ZFS "plugin" downloads the last ZFS version from OpenZFS github, so the functionality is there.

The ZFS plugin checks on every reboot for a new version from ZFS, downloads and installs it, as long as you have a active Internet connection on boot (don't forget if you run for example PiHole on Unraid you don't have a active Internet connection because DNS Resolution won't work unless you set a static DNS server for Unraid from Google or something else).

 

BTW The plugin is compiled from the official OpenZFS GitHub for each individual Unraid version, then the compiled package is uploaded to the Unraid ZFS Plugin GitHub and the plugin pulls it from there.

 

 

On 10/27/2022 at 3:58 AM, Sally-san said:

If it is part of the Unraid ZFS plugin already, how do we use it. I'm still a newb.

If it's integrated in the latest ZFS release (2.1.6) you can use it as usual on every other distro.

 

You can always get your ZFS version if you issue this command from a Unraid terminal:

zfs version

 

Link to comment

A quick note for ZFS users with NVMe devices.

 

While scrolling the OpenZFS github page, I've discovered this PR from Brian Atkinson concerning the Direct IO implementation within ZFS.

 

Not applicable for databases, but many others situations shall see a real benefit with its implementation.

 

You must understand that we're bypassing the ARC, and Direct I/O is very specific. So it's not a "performance booster" but a property that should be set accordingly to your need.

 

https://docs.google.com/presentation/d/1f9bE1S6KqwHWVJtsOOfCu_cVKAFQO94h/edit#slide=id.p1

Edited by gyto6
Link to comment

Hi everyone,

 

I have an odd issue that started with the new Unraid update 6.11.3.  On one of my 2 Unraid pro servers, the ZFS pool fails to "mount" (aka is not imported) on reboot with no error indicated.  A manual "zpool import poolName" works, but upon a subsequent reboot the ZFS pool is again "unmounted".  The second ZFS pool on the 2nd Unraid server does not exhibit this behavior.

 

"zpool status" shows everything is fine after I import.

 

I'm at a loss as to how to get the zpool to "auto import" on reboot.

 

Sorry for my poor terminology.  I hope I've explained the situation well enough.

ur-diagnostics-20221108-2238.zip

Edited by calvados
Link to comment
12 minutes ago, calvados said:

Sorry for my poor terminology.  I hope I've explained the situation well enough.

Your ZFS plugin is not installed properly...

 

Please run the following command from a Unraid terminal and then download the ZFS plugin again from the CA App:

rm -rf /boot/config/plugins/unRAID6-ZFS

 

grafik.png.0e61e6d20c49e303ea21c990aa3b34e3.png

  • Like 2
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.