dnoyeb

sweet, thanks. anyone tried out the new transcoder for hardware encoding / decoding yet?

Quick question, I am guessing that "latest" version is only pulling from beta. Any way to get the docker to update into 1.16.7.1597 instead? Curious to try out the new transcoder.

I too agree it would be nice to fork (not completely redirect) the unraid log to a secondary log server... Haven't ever seen a way to do that though.

yea, they've changed docker up a bit so i'm guessing my old directions are no longer valid. I have since moved on to using Splunk as a VM since I find that I don't come close to the 500mb of data a day limit for free use.

256 GB Multi-bit ECC

I kicked it over to 2048 which seems to work well. I've read that you should do it in multiples of 1536 however, so in Gparted if you kicked it to 3mb, you'd be set. If I pull the drive out again, I'll probably kick it to 3mb instead. I've got some more details over in that other thread (unassigned devices) showing how much it impacted my copies. I'm EXTREMELY happy now with the performance of doing large scale downloads... I'll probably test with 4x70gb downloads today and see how it handles it.... I'm on gigabit, so i'll routinely pull down at full gigabit; which really is a great test.

Haha yea... I've got a great flow now; files get downloaded to a Samsung 840 evo xfs formatted with the proper alignment which is mounted via unassigned devices as the temporary location; then they're extracted to the cache 1tb sandisk xfs drive (which works with the default 64 sector offset). I went from issues where extracting a 70gb file would spike load to 15+ down to around 4-5. Amazing how much faster and more efficient back to back downloads are now too. Pulled down 3 70GB files all back to back with repairs and unencryption; which normally would have thrashed my system.. It didn't break a sweat. System is a 32 core dual xeon with 256gb of ram; so do keep that in mind too.

Yea, I filled a feature request on it. Until then I am swapping over to a single SSD and just doing nightly backups of all of my docker data and will try and do a weekly backup of the VM's. One thing I noticed when moving down to the single SSD on XFS; load averages while moving say, two 70GB files back to back from another SSD to it; my load is hitting max of 5. When it was a BTRFS pool of two SSD's it was hitting 15-16. IOwait levels staying much lower as well.

Seems like something that should be variable for the cache drives using BTRFS. Anyone know where we file a feature request?

This isn't a requirement of BTRFS raid... I'm seeing where people use other starting blocks; so I guess this is an issue with unraid honestly and it forcing us to use sector 64. I'm still going to play around and see what happens; since EVERYONE was so positive that changing the starting block wouldn't have any impact on my other tests and I was able to reproduce it 4 times with 2 different samsung devices...

So ... Samsung devices have a different NAND Erase block size; which I think is causing us all some extra headaches. The starting block 64 isn't right for them and thats how they get partitioned in unraid and unassigned devices. Check my findings from that thread: The testings in the link above are to a single 500GB and a 250GB samsung 840. Both cleaned up after changing the starting block. Do keep in mind I have a unique setup with 256gb of ram; so my writes are in large blocks from the ram cache. As for the BTRFS pool... I've mixed in a non-samsung (SANDISK) SSD into my btrfs pool and it's really interesting to see how the two drives work differently. The Samsung will start causing huge IO wait times and it'll continue to write to the samsung upwards of 10 minutes after the sandisk is finished. I am going to try and use GPARTED to move the starting block over on the samsung and see if unraid will let it still work in the btrfs pool. I'll report back hopefully later tonight on that.

So, got some more SSD's; an OCZ and a Samsung 840. The Samsung exhibits the same exact issue as the other Samsung; the OCZ does not. I've done some reading and the Samsung 840 and 850 have a different NAND erase block size compared to most SSD manufactures: 1536kb and NAND page size: 8kb. So, starting off at 32k seems to cause issues for the Samsung drives. Before alignment: After alignment:

in the interest of testing; I don't want to change more than one variable at a time. I may play around with those settings after testing to see if they help level out writes however. One of my co-workers is letting me borrow a couple SSD's later today; one of them is another 840 samsung, and one is another brand. Will be an interesting test to see if I can reproduce the same outcomes.

I'll do the testing leaving everything the same at first; I was simply pointing out that the ram is probably part of why the issue doesn't manifest itself for most others (if this is actually an issue; trying to find another SSD now to test with).

So when you run the align-check and it comes back no, you're saying it really is... interesting. I'm going to throw another drive in and do the same test today for the fun of it. I think there is more to this honestly; I wouldn't be so quick to dismiss it. One other item of note that I should have mentioned that could be why it takes a while to manifest itself; the system has 256gb of ram in it. The testing was done multiple times on the same drive; after multiple reformats, different file systems (btrfs and xfs). This first picture shows the SSD writes on the bottom prior to the alignment; notice the bursts to write; then it starts to do smaller constant levels. This second picture is on the timeline to the right, as you'll see the transfers stay consistently slow. After the reads complete, the system will crap out after a while. This last picture is post alignment, notice a much smoother pattern of writes: The writes stayed looking like that until the entire 130GB was completed (two files, 70gb and 60gb in size back to back, the jump in read speed was as it moved from file to file). Anyways, I am going to find another SSD and throw it in and let UD format it and verify if the alignment is off per the align-check and if so; will do the same tests.

The first time around, yes. When I put it into the other box and used gparted; I repartitioned and formatted it xfs via gparted. Pretty easy for anyone to check and see if their SSD's are aligned or not using the parted command; will be curious to see what other's are getting results wise.

Not sure if anyone has run into this; but my symptoms were the following: 500GB SSD formatted xfs via unassigned devices, when performing LARGE constant file transfers TO the drive (writes) the performance would be great for the first 80GB of a constant transfer but shortly after 80GB the performance would plummet and the drive would almost become not useable. Thought it was the firmware as this is an 840EVO; so updated that, tried again; same issue. One of my co-workers remembered a long time ago that SSD's needed a certain alignment for the best performance. I read through some white papers from IBM on theirs as well; and it was recommended to check alignment via "parted" command align-check opt n Where n is the partition. I kept getting that it was not aligned. I pulled the SSD and stuck it in another box, booted a live gparted OS and redid the partitions while aligning them over to 1MB and verifiied via "align-check opt n" while still mounted in the gparted live OS. Pulled the drive, put it in the unraid box, booted and mounted it via unassigned. Ran the align-check against it and it still said aligned. Performed the same tests; copying two large files totalling 130GB to the now aligned SSD; and have no issues. Performance never drops off and all is well. So; not sure why the alignment wasn't right when done through unassigned; but figured I'd let everyone know my experience and the fix in case they ran into this.

i'm actually having the same issue.... trying to remove a partition / drive so I can change the format from BTRFS to XFS and am getting the same behavior. I am on the latest unraid (6.4.1 and the up to date unassigned devices) Feb 19 21:52:36 unRAID unassigned.devices: Device '/dev/sdb1' script file not found. 'REMOVE' script not executed. Feb 19 21:52:36 unRAID unassigned.devices: Unmounting '/dev/sdb1'... Feb 19 21:52:36 unRAID unassigned.devices: Successfully unmounted '/dev/sdb1' Feb 19 21:52:47 unRAID unassigned.devices: Removing partition '1' from disk '/dev/sdb'. Seems it doesn't have a remove script for it for some reason

Me and one of the guys at work went through the process to get this working and figured we'd share since there isn't an unraid community apps setup done for this... First up, make sure you turn on the Docker hub search feature for community apps... You'll install three seperate dockers, all outlined below... MongoDB Docker setup Add MongoDB from Community apps change name for MongoDB docker to some-mongo Elasticsearch Docker setup Add Elasticsearch (official) from docker hub Change Repository for Elastic to: elasticsearch:2 elasticsearch -Des.cluster.name="graylog" change name for Elasticsearch docker to some-elasticsearch Set path /mnt/user/appdata/graylog/data/elasticsearch for /usr/share/elasticsearch/data Graylog2 Docker Setup Install graylog2 from DockerHUB (graylog2/server) Put these in extra parameters: --link some-mongo:mongo --link some-elasticsearch:elasticsearch create variable with key= GRAYLOG_WEB_ENDPOINT_URI and set key’s value to = http://127.0.0.1:9000/api Add TCP port 9000 add UDP port 514 add UDP port 12201 Set path /mnt/user/appdata/graylog/data/journal for /usr/share/graylog/data/journal Set path /mnt/user/appdata/graylog/config for /usr/share/graylog/data/config On unraid box, cd into your /mnt/user/appdata/graylog/config folder and run the following two commands: wget https://raw.githubusercontent.com/Graylog2/graylog2-images/2.1/docker/config/graylog.conf wget https://raw.githubusercontent.com/Graylog2/graylog2-images/2.1/docker/config/log4j2.xml Anyways, after doing that, you'll have the three dockers all communicating at each other... you have to setup an input in Graylog (I use syslog UDP 514) and point your various servers at it... Going to play more with getting all the other dockers to dump their syslogs into that setup next and will update the post once I do.

256gb registered ecc ram too much?

Just wanted to check in and see if there were plans to update the unifi wifi and unifi video dockers to the latest versions?

i'm not sure this is still working... I swear it was working fine back on 6.1.1 but on 6.1.4 and now 6.1.6, on an app like Sabnzbd which I have limited to "--cpuset-cpus=0,1" I'll see it spiking all 4 cores on my i7 (as well as the hyperthreaded cores) and I'll even have issues where in sab it'll show loads upwards of 20 and it'll impact another docker that is assigned 2-5 to the point the other docker loses connections with the cameras (camera server) Is there some better way to tell if it really is still working like it should?

sadly I don't think this made it in the 6.1.4 release looks like i'll be building from scratch again I guess.

I have this same issue. Only happening on TrueHD and DTS-HD. Would love this fixed. Well at least i'm not alone! whew! makes me feel better... Maybe the 6.0 release has some new drivers or something in it, cannot wait to try it out. Probably wait for the unraid official instead of installing it from scratch this time around though

Just curious, to any of you guys running nvidia hardware I have a question. Have you tried watching any TrueHD or DTS HD Master audio 7.1 movies? I find that after I put in the "options snd-hda-intel enable_msi=1" in the config file; I can watch all my movies flawlessly that are just normal old DTS or DD 5.1... However anytime I watch a FULL movie in either TrueHD or DTS HD Master audio that about every 15-20 minutes the audio will cut out for about 1-2 seconds then come right back. Anyways, it only happens on that specific setup... anyone else have any experience with it? I've gone as far as building a custom openelec vm and it has the same issue; so not just the one that is built into unRaids... I do have my Vid card in a slower PCIex lane as I only had one 16x lane, but before I go building an all new unraid box; I figured i'd see if that'll be worth it... Not CPU related, cpu is sitting on its rear...