gilahacker

I formatted a single 4 TB drive in my array as BTRFS and enabled compression using `chattr -c`. I also installed the "compsize" tool, to show me what kind of compression I was getting: Processed 213570 files, 1597070 regular extents (1597070 refs), 46817 inline, 223456 fragments. Type Perc Disk Usage Uncompressed Referenced TOTAL 96% 1.8T 1.8T 1.8T none 100% 1.7T 1.7T 1.7T zlib 43% 50G 115G 115G Out of 1.8 TB total data, only 115 GB was compressed. It was getting a good compression ratio (2.3:1). I wanted to increase the compression level, but couldn't find any way to do that since there's no entry for the drive in /etc/fstab like there would be on a non-unRAID setup. While reading up on that, I discovered zstd (even though it's been around for a while now) and that seemed like a better option than the default zlib, so I ran btrfs filesystem defragment -rf -czstd /mnt/disk18 to switch to zstd and it compressed a lot more of the files: Processed 149606 files, 12382461 regular extents (12382461 refs), 29274 inline, 781994 fragments. Type Perc Disk Usage Uncompressed Referenced TOTAL 73% 1.2T 1.7T 1.7T none 100% 405G 405G 405G zstd 65% 915G 1.3T 1.3T (note that I deleted some unnecessary files after the defragment, so my total size decreased) The compression ratio dropped (1.45:1), but >10x more data is being compressed now so I'm "saving" ~416 GB of disk space whereas I was only saving 65GB before. Based on what I've found, zstd compression should be as good or better than zlib while being a whole lot faster. Increasing the compression level means it takes longer to initially compress, but decompression rate stays about the same (a trait zlib shares). I'd love to be able to crank up the compression level, but the changes to allow that either aren't merged into the btrfs command line tool yet or the version we have isn't up to date. I found lots of discussion about adding them on github, but haven't compared version numbers or anything. There's also supposedly a way to force compression on all files as, by default, btrfs only tries to compress the beginning of the file and if it doesn't appear to compress well it just leaves the whole file uncompressed. This is why even after running the defrag command above, 405 GB is still listed as having no compression. I haven't figured out if there's a way to do that yet. In my particular case, many of the files are already RAR or ZIP or whatever, which probably wouldn't compress well anyways, but I've been slowly crawling through and unpacking those files so I can actually browse their contents in my file manager. This drive contains my collection of STL files for 3D printing which take up the majority of the space and a few old system backups that take up 158 GB.

Then one of us misunderstood what @Zer0Nin3r was asking about regarding `copy --reflink`, which requires the `reflink=1` flag to be set when the disk is formatted.

I just added a new disk to my array and found out about the reflink thing while trying to figure out exactly why it shows 66 GB used* on an empty, newly formatted 10 TB disk. All of my old disks have reflink=0 (per xfs_info command), and I don't believe it's possible to enable it without a reformat. *Seemed high, but I'm honestly not sure what it was on other disks when I added them. Something I stumbled upon in a Google search indicated that new XFS disks have significantly more "used" space to start with when that feature is enabled.

@Hoopster brings up an interesting point. Because, unlike many (most?) Linux distributions, unRAID runs completely within RAM and nothing else gets mounted over-top the default rootfs (special instance of tmpfs). I believe this is similar to any "Live" distro, but don't have experience with those. So even though my /tmp isn't a tmpfs mount itself, it is *on* a tmpfs mount and exists only within RAM. Thus, either /tmp (a path on a tmpfs mount) or /dev/shm (an explicit tmpfs mount) should work exactly the same other than the fact that some of the space on / will be in use already.

On *my* server, running 6.6.6, I have a /dev/shm tmpfs mount that is allocated 32 GB of RAM (per df). I have 64 GB total RAM, not sure why it's set to 32. I do not have a /tmp mount. I do have a /tmp directory, but it's not a tmpfs "RAM drive", just a plain directory. You can run cat /etc/mtab to see what your current mount situation is. I imagine it's similar. I have the Plex /transcode folder mapped to /dev/shm in its Docker settings. No issues so far.

Cron format: # .---------------- minute (0 - 59) # | .------------- hour (0 - 23) # | | .---------- day of month (1 - 31) # | | | .------- month (1 - 12) OR jan,feb,mar,apr ... # | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat # | | | | | # * * * * * user-name command to be executed For 0 0 * * 1,3,5 it would run at midnight every Monday, Wednesday, and Friday. https://crontab.guru is great for figuring out cron schedules. But, as @ljm42 said, you might want to use the CA User Scripts plugin instead.

Now I really want to try this once I have stupid amounts of RAM, even if it's just to see if I can make it work.

Finally got this container working just now. Had tried when I first started using unRAID about a month or two ago and did something wrong, who knows what. Blew away existing config, reinstalled with proper path and port mapping, and it's doing its thing as I type this message. It hasn't gotten far enough along the current disk to say for sure, but it looks like it's going about as fast as was when I had this drive in the Windows machine that unRAID is replacing. I understand that Docker isn't really supposed to add any overhead, but you never know what the differences are going to be between two completely different systems. Now to see how long it takes to go through the pile of DVDs and Blu-rays in my closet. :-P Thanks Saarg!

+1 Is there a way of making this an official feature request? Okay, forum n00b just realized this *is* the way you officially request features around here...

I can hardly wait. :-)

16 GB right now, which is significantly more than all but one of my current video files (because I haven't reencoded it to a reasonable size yet). I did just get a 4k TV and an Nvidia Shield Android TV, so I *might* try out some 4k videos. Planning on getting 128 GB when I upgrade later this year (yes, partially just because I can). I know I'm just being paranoid about the SSD. I went with an 850 Evo instead of the Pro and, IIRC, it's rated for half as many writes by Samsung (but will likely still outlive all my spindle drives). Still... there's really no reason *not* to use the RAM if it's available. If the server crashes/loses power/whatever then that data doesn't matter anyways.

My SSD is plenty fast, but transcoding in RAM would be nice just to avoid unnecessary wear on the SSD for those of us who have enough RAM.

*UPDATE* Nevermind, it just took *forever* to become accessible. GUI is working, streaming is working. Everything is good and right in the world. :-) Original post: Updated my Docker today (first time I've done that since setting up the server about a week ago) and now it doesn't work. unRAID shows it as running, but GUI is not accessible and the log shows it's just sitting at "Starting Avahi daemon". Using linuxserver/plex.latest. :-(