MrCrispy

but this is not true. even standard SATA has much higher write speeds. There are posts here where people have done plenty of tests and the conclusion was the smb and FUSE in unraid was to blame, without that they got much higher speeds, without using any cache.

you just made 2 completely contradictory statements. So do you think its designed for home or for business users? I think your 2nd sentence, you meant home? The speed issue is a real problem, I don't know how much of that is due to FUSE or due to Unraid's implementation. Looking at other FUSE implementations and tests, it seems to be the latter. I have asked about it, but no technical responses. A cache pool is just mitigating, not looking at the source of the problem. Maybe some small mom and pop shops use Unraid. A business wouldn't. They don't need the additional features for vm's, dockers, CA etc. The primary use case is as a file server. For this the main benefit Unraid has is mix and match existing disks, which is not a concern for them.

this is precisely what I am concerned about. Hence why I suggested that existing Pro users should be upgraded to Lifetime and not further distinctions made. If an old legacy basic/plus upgrades later to Pro, they should also be converted to Lifetime. The reasons given for not doing this are for accounting/metrics, that may be true but it doesn't sit well with me.

just a small quibblle - I honestly don't understand the love for Proxmox. It doesn't even have docker support, you are supposed to run a Linux vm instance for those. All it does is vm's, zfs/storage pools etc. Is running multiple vm's that popular? isn't that exactly what docker does, much more efficiently, for most cases where you needed a vm before. And if you do need vm's, whats wrong with qemu+kvm. Proxmox just adds a hypervisor for added complexity that IMO is completely pointless for home users. Also like I said before none of these do disk pooling with native data format. I think people don't realize unaware most people are, they look at competing RAID solutions and think its the same thing.

Any improvements to the core system will be welcome. Unfortunately I don't think there have been any? all the new stuff is about dockers/vm/themes? I posted before about my thoughts on this, using snapraid etc, but have not gotten any official replies, everything Ive seen is only about ZFS

For people with massive servers, the $250 price is just 1 more hdd out of the 20 they have, so its no big deal. for casual users, its the choice between adding a 20TB hdd or not. Remember if they use Unraid they also need another hdd of same size for parity. In the real world a lot of people don't use backups or parity, this is not a good thing of course but its the reality. This is where the new price will hurt.

I think the value proposition of Unraid is not being communicated to users and is getting lost. Unraid is primarily about the array - i.e. shfs using software real time parity, mix/match disk sizes, and NOT striping data. TrueNas or any of the custom NAS OS's don't do this. THIS is what keeps your data safe and allows you to grow. You can use mergerfs to combine disks. But the only alternative for parity is snapraid and its not real time. Everything else - vm's, dockers, community apps. all of that came much later, all of it its just a wrapper around open source projects you can use on any other distro or custom OS, with maybe a little bit more work. No one else has the data safefy. Not if you want to avoid striping and keep data in native format.

I agree with this, but also the industry seems to be moving in this direction (i.e. subscriptions) and in general anti consumer practices. e.g rentals for movies, not owning anything, devices that can't be repaired, digital content you purchased for 'lifetime' that just disappears or stops working. I read that big companies like Plex are going to discontinue lifetime licenses because its not a good business model. And I'm sure Plex makes quite a bit of money. I'm going to say something that is obviously not the direction LimeTech wants, so please don't take it that I am advocating or asking for this: in my ideal scenario, Unraid goes full open source! Along with that comes massive influx of developers and much faster pace of development (as we've seen, the community is probably a massive part of it already). LT makes money by offering support, value add features, premium addons, partners with hardware oem's. As well as offer other products, integrations that other businesses will pay for etc. I'm not a bizdev person, all this may be unfeasible. I wonder how many people will pay for lifetime vs the annual fees? or how often updates will be needed? I think LT giving all these options and looking out for current users is so admirable, I do wish there was a clear revenue stream that also allows people who can't afford the new prices to use it.

I understand keeping old legacy Pro and Lifetime users separate, for metrics and measuring etc. but it still leaves the door open for future changes that differentiate between the two. It would've been nicer if there's a single Lifetime license type, and current users seamlessly get upgraded to it.

If speed is a priority and you want data striping, IMHO Unraid is not the product for you. There are any number of free and commercial RAID based solutions that can accomplish this. As to why Limetech is spending money on ZFS while seemingly completely ignoring Snapraid like functionality, I believe the answer is what I said earlier - ZFS is a very big name and can bring in popularity and $$$, and many of the big Unraid adopters and popularizers are people with $$$ and giant servers, youtube channels, who can afford it. ZFS makes almost no sense for the typical home Unraid user, vs the ones with giant server racks and multiple servers and enterprise gear.

Yes, I believe its the cse-ptjbod-cb2 motherboard. My qn was more about how to turn the DAS on/off along with the main server, and does it go to sleep? So you will have close to 600TB of storage? that is one hell of a server!

I believe ZFS ideally needs a lot of RAM. As well as higher resource requirements in general. ZFS will use half of RAM as cache and upto 90% if needed. A quote from another forum - "ZFS is memory hungry, more than other file systems. Given the way it writes data to disk (non-linear, log-structured), it tends to have a large amount of metadata in memory, and make heavy use of that metadata. If that metadata gets corrupted in memory, bad things will happen. An even more important argument is the following: ZFS always reads and writes CRCs, so the data on disk and on the way to/from the disk is safe against corruption. That means that data in memory is now the largest target for corruption. While this doesn't mean that ECC is more necessary than for other filesystems (which also fill all unused memory with cache data), for ZFS the relative importance of memory errors is higher" SnapRaid as an external tool would not make any use of unRaid. The shfs code already has every write/read to the array go thru it. It doesn't make sense to have to run an external tool like snapRaid sync to read data and check parity, when that is already being done inside unraid. What could be added is checksums, as well as other functionality, in the same driver in Unraid that computes and checks parity. Then its a simple matter of using snapraid's other tools and utils. Unless we hear from the official team, this isn't going to go anywhere. Sadly I don't think anyone else cares about this, all the hype is about ZFS, and its far easier to sell ZFS as a new feature. There is a big inertia in the Unraid community, the few threads I've read on this all basically amount to 'i'm happy with what I have and we shouldn't demand more'. The file integrity plugin is already deprecated I believe and in any case would not prevent against bitrot.

I agree with you. Alas it seems ZFS has a very large mind share and is considered some kind of uber-solution, which I don't agree with at all. Its resource hungry, enterprise focused, and brings nothing to the table as far as I can tell vs btrfs, except a better name. And of course, ZFS/btrfs and any other form of striping is a complete non starter for a home solution, which is why unRaid exists in the first place. Is it really too much to ask to integrate snapRaid style checksumming/integrity checks into shfs? It would bring a huge host of benefits to everyone. Those who want to run enterprise style pools with ZFS and spend $$$$ on memory, drives etc, are welcome. But honestly, why not run TrueNas or plain old Linux if thats what you want, since the only thing lacking would be the docker integration and the CA app store, which honestly can be added manually quite easily.

Looking at a similar SM chassis. Does anyone run these as a JBOD DAS? I want to use these with an existing server to add more storage. When you connect a DAS like this how does it turn on/off?

I'm planning to use a DAS, which will be an external JBOD enclosure connected to an LSI HBA on the server, with Unraid. Does anyone use a setup like this? - do you keep the DAS powered on 24/7? does it go to sleep? The drives themselves can sleep if idle, right? - is there a way to turn it on/off with the server? - are there any plugins/tools in Unraid for this? - do the drives show up as regular internal hdd's? do you get full SMART info etc?

There are plenty of threads about user shares being slow and esp in conjunction with smb. I also posted on Reddit asking for opinions and several people confirmed that the latest version is still very slow for access over SMB and esp for lots of files. So I understand that a FUSE filesystem like shfs has to have some overhead, but not this much. unRaid also officially recommends using a disk share or exclusive access for higher speeds, which to me defeats the entire point, as I'd like to use user shares backed by parity. what is the overhead of sfhs vs say mergerfs? there does seem to be an issue with smb implementation, plenty of technical details posted in above threads. 100-1000x slower is not good if true is the problem just with shfs, smb, or a combination of both? e.g. I saw some say that sftp to user shares was much faster is this being looked at and are there any plans? plugins to cache dirs are not really fixing the core issue, and even they don't seem to work

The discussion is about using features similar to Snapraid, not to discuss that product. Calling it vaporware is not called for, it seems to be a very well designed and regarded product from a technical perspective. I presented several benefits over Unraid and shfs. as well as the compromises made. The discussion is about those, as well as thoughts on ZFS. such can be said for other systems like unRaid too. or you could say that Snapraid gives you additional features like undepete, snapshots for specific folders, bitrot protection, and offline parity. there are many many ways to slice and dice and improve. Just dismissing other approaches isn't good. I know this is an Unraid forum but I was hoping for a discussion on technical merits. FWIW I own a Pro key, its not like I'm not aware of the benefits.

After watching the recent interview as well as the general direction, it seems the intention is to have ZFS be the new filesystem for arrays in Unraid? If so, I am rather hesitant, because even though I'm sure it will not be mandatory, I still don't think ZFS is a good fit. There are a lot of ZFS fans used to enterprise level hw and sw and obviously it has many great things, but IMO it fundamentally doesn't really follow the basis ethos and tenets of a home server fs - - data disks should remain in native format and independently readable - only the disk with data needed spins up - you can mix and match disks of any size The basic issue is that a striping implementation such as traditional Raid, ZFS or btrfs can have none of these. And ZFS is also notoriously 'heavy', it will eat up ram, it needs bigger caches etc. Honestly I think for its intended audience, btrfs is a much better solution, except it doesn't have the mind share or history. The other big feature of course is checksumming, snapshots etc, totally lacking in Unraid. I think the best way to add this would be to be pragmatic - doing these in a realtime system is impossible without block level striping. What Snapraid does is IMO perfect - periodic checksums, snapshots, backups, bitrot protection, undelete, dedup and so many other features become possible with very little overhead, with the only caveat being its not realtime - which if you think about it is actually better for an array since most media is RO anyway. I know this has been discussed before but there was never any real conclusion and it would be nice to get a fresh discussion and official thoughts?

what do you use to encode - CPU/hw?

I know that encoding times/quality is dependent on many factors, but can someone give me a general idea of what I can expect for converting my library to hevc. Assume average bitrates for the input media and the default Tdarr setup. - how much time would it take to convert a 1hr video to h265 - what kind of reduction to expect? - input media ranges from SD to 1080p. Is it worthwhile to convert SD/480p or just leave them alone? - if I get an Nvidia card, can I then run 2 nodes on same machine, one using Intel, other one the gpu? Right now I have an Intel 8th gen cpu for QuickSync. I know Nvenc will be faster. I've also read there is a drop in quality which can be compensated by an increase in output size? Is this correct? Would just like to get a general idea. The 2 cards I am looking at are the Quadro P400 or a 1050 Ti, what is the difference in encoding speed between them (in general does a card costing 2x encode 2x as fast?)

Thank you, that answers my qn. I am assuming you have the default setup which replaces files as they are encoded?

Right, but all the guides recommend using a fast drive like ssd for this. Its at the very least a different folder, so my qn remains the same.

I'd read in another thread that as Tdarr executes each plugin it copies the file to its temp/working folder (which is on cache), writes the output file, which is then copied to the original location, then the next plugin is run and repeats the copies so on, and someone had replied that this was going to be fixed later so that all plugins run on intermediate file. Is this still the case?

Thats what I thought. so with their setting, any directory can be split across disks. with the benefit of hard links. So would it be correct to say that an unRaid share is a FUSE file system and thus hard links cannot work across shares? Is this a popular setup? do people generally use different split levels and multiple shares? would like to hear some opinions.

The well known trash guides recommend a single share with split level = 'automatically split any directory' as shown here - https://trash-guides.info/Hardlinks/How-to-setup-for/Unraid/#__tabbed_2_1 I understand they've done this for maximum flexibility. A lot of people prefer keeping tv shows or at least seasons on the same disk and definitely movies. Is there any way to use other split strategies with their setup?