falconexe

Ultimate UNRAID Dashboard (UUD) Current Release: UUD 1.7 (2024-03-30) UUD NEWS: 2024-03-30: UUD Version 1.7 is Released 2022-06-19: The UUD Forum Topic Reaches 200,000 Views! 👀👀👀 2022-04-17: The UUD Tops 5,000 Unique Downloads 💾💾💾 🎉 2021-05-26: The UUD Forum Topic Reaches 1,000 Replies! 2021-04-17: The UUD Forum Topic Reaches 100,000 Views! 👀👀 2021-03-26: The UUD Tops 2,500 Unique Downloads 💾💾 🎉 2021-03-23: UUD 1.6 is Featured Again in the "Best of the Forum" Blog 🥇🥇 2021-03-21: The UUD Forum Topic Reaches 75,000 Views! 👀 2021-03-20: UUD Version 1.6 is Released 2021-01-19: The UUD Forum Topic Reaches 50,000 Views! 👀 2021-01-11: The UUD Tops 1,000 Unique Downloads 💾 🎉 2021-01-07: UUD is Featured as the FIRST "Best of the Forum" Blog 🥇 2021-01-06: UUD Donations Site is Created 2020-12-31: UUD Version 1.5 is Released 2020-10-09: UUD Version 1.4 is Released 2020-09-28: The UUD is Featured in the Official UNRAID Monthly Newsletter (September 2020)! 2020-09-21: UUD Version 1.3 is Released 2020-09-14: UUD Version 1.2 is Released 2020-09-12: UUD Version 1.1 is Released 2020-09-11: The UUD is Born and Version 1.0 is Released Overview: Welcome to the OFFICIAL UUD forum topic. The UUD is my attempt to develop the Ultimate Grafana/Telegraf/InfluxDB/Plex/Tautulli/Varken dashboard. This entire endeavor started when one of our fellow users @hermy65 posed a simple, but complex question in another forum topic (see post #3). I decided to give it a shot, as I am an IT professional, specifically in enterprise data warehouse/SQL server. After a few days of hard work, UUD version 1.0 was released. We are currently on Version 1.6, and the project is in active development. If you are a Grafana developer, or have had experience building dashboards/panels for UNRAID, please let me know. I would love to collaborate. Version 1.7 Screenshots (Click the Images as They are Very High Resolution): Disclaimer: This is based on my 30 Drive UNRAID Array. So this shows an example of a fully maxed out UNRAID setup with max drives, dual CPUs, Dual NICs, etc. You will/may need to adjust panels & queries to accommodate your individual UNRAID/PLEX architecture. I have spent many hours custom coding new functionality and features based on that original template. Much has been learned and I am excited to see how far this can go in the future. Thanks again! Developers: Primary Developer: @falconexe (USA) UUD Creator | Active Development | Panels | Database Queries | Integration | Look & Feel | GUI | Refinement | Support Developer/Colleague: @GilbN (Europe) Dashboard Examples | Back-end | Dynamics | REGEX | Support | Tutorials Contributors: @hermy65 @atribe @Roxedus @SpencerJ @testdasi @ChatNoir @MammothJerk @FreeMan @danktankk @Dazog @MrLondon @LTM @mattekure @ptchernegovski @caplam @RockDawg @corgan @jbartlett @Hoopster @LTM @skaterpunk0187 @valiente @Agent531C @JudMeherg @Jturner @1471 Dependencies (Last Updated On 2021-03-20) Docker - InfluxDB Docker - Telegraf Docker Network Type: HOST (Otherwise You May Not Get All Server Metrics) 👉 Create Telegraf Configuration File 👈 (DO THIS FIRST!) Create and Place a File into Directory "mnt/user/appdata/YOUR_TELEGRAF_FOLDER" Enable and Install Telegraf Plugins Telegraf Plugin - [[inputs.net]] Enable in telegraf.config Telegraf Plugin - [[inputs.docker]] Enable in telegraf.config Telegraf Plugin - [[inputs.diskio]] Enable in telegraf.config To Use Static Drive Serial Numbers in Grafana (For DiskIO Queries) Do the Following: Edit telegraf.conf > [[inputs.diskio]] > Add device_tags = ["ID_SERIAL"] > Use ID_SERIAL Flag in Grafana Now Upon Booting, You Don't Have to Worry About SD* Mounts Changing (So Your Graphs Don't Get Messed Up!) You Can Also Set Overrides on the Query Fields to Map the Serial Number to a Common Disk Name Like "DISK01" etc. Telegraf Plugin - [[inputs.smart]] Enable in telegraf.config Also Enable "attributes = true" Bash Into Telegraf Docker and Run "apk add smartmontools" Telegraf Plugin - [[inputs.ipmi_sensor]] Enable in telegraf.config Bash Into Telegraf Docker and Run "apk add ipmitool" Telegraf Plugin - [[inputs.apcupsd]] Enable in telegraf.config Telegraf Docker Config Add New Path (NOTE: This path has now been merged into Atribe's Telegraf Docker Image. (Thanks @GilbN & @atribe) Post Arguments "/bin/sh -c 'apk update && apk upgrade && apk add ipmitool && apk add smartmontools && telegraf'" Docker - Grafana Grafana Plugins Pie Chart Panel Run Following Command in Docker: grafana-cli plugins install grafana-piechart-panel World Map Run Following Command in Docker: grafana-cli plugins install grafana-worldmap-panel JSON API Run Following Command in Docker: grafana-cli plugins install marcusolsson-json-datasource Dynamic Image Panel Run Following Command in Docker: grafana-cli plugins install dalvany-image-panel Docker - Tautulli Docker - Varken CA Plugin: IPMI Tools License: GeoLite2 (Free) NON SERVER HARDWARE (If You Cannot Use "IPMI" and Need to Use "Sensors") As an alternate to IPMI to monitor CPU/System/Aux Temps, you can try the Sensors Plugin. Telegraf Plugin - [[inputs.sensors]] Enable in the Telegraf Config (Uncomment It) Bash into the Telegraf Docker and Execute "apk add lm_sensors" Stop All 3 Dockers (Grafana > Telegraf > InfluxDB) If You Want to Keep This Plugin in Perpetuity, You Will Need to Modify Your Telegraf Docker Post Arguments (Adding lm_sensors): "/bin/sh -c 'apk update && apk upgrade && apk add ipmitool && apk add smartmontools && apk add lm_sensors && telegraf'" Start All 3 Dockers (InfluxDB > Telegraf > Grafana) Dashboard Variables (Update These For Your Server): I haven't been this geeked out about my UNRAID server in a very long time. This is the cherry on top for my UNRAID experience going back to 2014 when I built my first server. Thanks everyone! VERSION 1.7 (Current) Ultimate UNRAID Dashboard - Version 1.7 - 2024-03-30 (falconexe).json @SpencerJ

Thanks for this tip. I've set it up like this now.

Thanks. I'll check it out.

So, Duplicati is terrible. I couldn't even get my initial 7TB backup set to complete due to all sorts of database/block issues. Basically, if you interrupt the initial set in any way (like graceful pausing or stopping after current file is finished ***BROKEN***, the entire thing fails. I tried 7 times to get it done, and each time some kind of fatal exception would occur. I uploaded 2TB at one point, only to have it fail once I stopped it (gracefully) per the documentation, in the hopes that I could resume it later. After reading many posts in their own forums, many others have had issues getting very large initial backup sets completed. I was hopeful, but now just depressed ha ha. Now I'm on to the paid Duplicacy. So far, not really impressed, but I'm trying the same 7TB backup set now. It does appear to be able to start/stop the initial set via a native partial backup snapshot function, so I am hopeful this product will succeed where the other free one failed. However, I am trying to restore a 30GB backup I did last night, and it is stuck on "Starting" for like 30 minutes, so.....not looking great. I'll report back my findings. At this point, my only other option if this fails is to try RClone, or go back to my Windows 10/Backblaze solution. @johnwhicker, I didn't want to believe that ALL of these options are nonviable, but you appear to be correct. Which completely SUCKS!

Interesting. Thanks for sharing.

@johnwhicker By the way, Love the name. I am a HUGE John Wick fan! "People keep asking me if I'm back! Well I'm thinking I'm BACKing up UNRAID the right way."

Hi, I've been holding off on replying to this topic because I've been in the same boat and have been trying to figure out a viable long term solution, and recently came across one that is working well for me so far. Once Crashplan was discontinued, I had given up on a native UNRAID cloud backup solution a few months back and went with BackBlaze. I was using a Windows 10 machine to sync my unRAID shares with software called "SyncFolders" and a fleet of USB3 portable drives outside of UNRAID. Needless to say, it was a bit much, but it worked. I recently came across both Rclone (file syncing ) and Duplicati (true block based backups). I tried both, and I ended up using Duplicati and a cloud based solution with encryption, specifically GSuite (Google Drive For Business). They currently offer unlimited storage for $12/Month, and they do not enforce the 5 user minimum. However, you do have to own your own domain to use this solution. I already had one, so no big deal. The only thing with Google Drive, is that you can only upload 750GB/Day. I hit exactly 725.4GB before Duplicati started throwing server side errors. I have since throttled my uploads to 8 MB/s to keep it under this ceiling. (Math works out to 691.2 GB/Day [8MB * 86,400 Sec / 1000MB = 691.2GB/Day]. 9 MB/s puts it over, and the parameter has to be a whole number). This should keep Duplicati happy and support uninterrupted backups during my initial upload set. This would never be a problem once all files are initially backed up, but it is an interesting facet of the this solution's workflow. Other than that, I have had no issues. Restoring works from my testing with various files types and sizes. I'll be testing a full 8TB restore here soon once the initial backup set is completed. Hopefully, I won't run into the issues you did, but I remain hopeful. I am interested to hear what specifically your issues were with Duplicati and restores. It looks like you have not tried Rclone yet, so it may be worth a shot. Here are some great tutorials by @SpaceInvaderOne. Great tutorial if you just want a straight forward encrypted cloud sync via Rclone: Here is another way of performing block based backups (true backups) not syncing with Duplicati. You can use the same cloud services, or even UNRAID to UNRAID. This is what I am currently using. Hopefully this helps you and/or others who will be going down this path. I do intend on using Duplicati to backup UNRAID to UNRAID as well. Let me know if you have any questions. Good Luck!

Triple Parity Please 🙏 I ask this half serious, and I know the math behind this would be insane. I'd take this or multiple arrays on single hardware. I have nearly 30 disks in a 45 Drives Storinator with 206TB usable, including X5 14TB drives. As single drive sizes go up, I'm getting more and more nervous ha ha. I have backups of everything, on and off site, but still, recovering would be a PITA.

I just Friggin hit 206TB usable. Approaching .25PB 😍

I just bought/built this insane rig. Dual 8 Core CPU, 30+ Devices, 200TB usable (still have one 14TB drive pre-clearing so those pics show 184TB). Meet "MassEffect". 45 Drives Storinator Q30 Turbo. Huge improvement over my old Norco 24 bay rig (now backup server). Rock solid temps peaking at 32C during parity/rebuilds. From the largest server poll thread, this may be 🤷‍♂️ the largest single UNRAID build yet (at least that I have seen documented) and completely pushes UNRAID to it's current limit on number of drives. I can take this thing all the way to 448TB with 28X16TB drives with dual parity if I want to get really crazy. Loving it!

And 4K HDR...😂

OK, cool. Interesting design with the extra 12 drives in the back. It looks almost identical to my Norco 4224 case. Thanks for sharing.

Also, 45 Drives offers multi-year RMA on RAM so that was huge selling point. Their support is top notch overall too. I opted out of the extended support contract because I work in the IT industry and have built countless gaming rigs and custom servers. Overall, I had no time, wanted the most painless and efficient build possible, and a rock solid company behind our new Production UNRAID server. You pay for that kind of quality and peace of mind and I am OK with the price. That is a SICK deal. What were the specs? Got any pics? I would love to see a chassis beyond 30. Thanks! Living in the USA, and with this shipping from Canada, I only had one minor hiccup at USA customs requiring a form to be filled out. It took about 8 hours to clear customs once that form was processed. I had to contact FedEx corporate to get that sorted out. Otherwise 45 Drives shipped this thing ridiculously quick and were super helpful overall. They also packed this thing very securely with custom foam and a sturdy box. The rails (super beefy) came in a separate box. Finally, the rails were super simple to install on the chassis and onto my StarTech 26U rack-mount, unlike the Norco rails😒. I was able to get it mounted all by myself. It was a bit tricky lining up both rails blind on one side, but I got it done LOL. Here is the ONE thing I have learned from this experience and purchase. LOWER DRIVE TEMPS ARE WORTH EVERY PENNY! It literally makes me sick to think of how hot I had been running my drives for the last 5 years (average around 45C and peaking at 52C). The catalyst for this purchase is that I came home to 3 drives hitting 62C in a data rebuild situation. I'm pretty sure I lost the 6TB drive I was rebuilding because of previous long term sustained high temps, and then I was on super thin ice attempting a rebuild with those temps (14TB drives run a bit hotter and I added 3). All of a sudden, My Norco case started losing cooling efficiency. No changes to the fan controllers, no fans failed, but we have had a very hot summer where I live. Once that 6TB drive failed, I jumped to extending Dual Parity to 14TB, and then swapped that failed drive to 14TB. Once the temps reached 62C and averaged 55C during that rebuild, I said NOPE. I stopped the rebuild, and ordered my Q30. Once it arrived, I rebuilt the same drive and had a peak of 32C. I have since done 5 parity checks and it is still ROCK SOLID on temps. This speaks immensely to the design and thought that 45 Drives puts into their products! Could I have upgraded my Norco case with better fans/cooling? Sure, but it would have never been this good due to the drive setup and orientation in that case. Replacing drives isn't cheap! When it comes to our data, even though I have redundant physical and online backups, the tranquility I have sleeping at night with this rig speaks for itself in my opinion.

I get it... My first Norco case (24 Bays) was around $2500 without drives 5 years back. It lasted me a long time until I outgrew it and heat became an issue as platter sizes increased. I look at it as a long term investment. We also run a media production company, so it is a business expense. That's the great thing about UNRAID. It is insanely customization, scalable, and can literally run on anything from potato to a Lamborghini. 😂

This Turbo model (Dual CPU and 64GB of RAM) retails for just under $6K with free shipping, and you get what you pay for. You can get into a Q30 starting in the high $3K range. You can use their Q30 configuration page below and price one out. Ask for Dylan! He was great to work with. https://www.45drives.com/products/storinator-q30-configurations.php

Nope! I used to when I segregated certain media on certain drives for certain dockers. But not since 2014. I do however user the Spin Down Delay.

X4 16GB Modules: 64 GB DDR4 Multi-bit ECC (4 of 8 Slots Used)

Alright here you go! This currently shows 186TB usable as I have my last 14TB disk pre-clearing in another UNRAID server but you get the idea. Once I install that into SLOT 28, I will be at the 200TB described. That leaves me with another 28TBs to expand with the last 2 empty slots at 14TB each for a total of 228TB usable, unless I step up to 16TB. However, I'll probably wait until the industry hits 20TB per disk to increase because I'm on dual parity so I'd have to purchase X3 drives to expand the size of any disk past parity. In the build below I added the 10GIG NIC and 2 PCI SSD trays for my Cache pool. Other than that, I added the drives to the array and the rest came from the factory (45 Drives). By the way, the friction mounts in this thing are friggen awesome. No. More. Screws/Trays! To fire this thing up I literally just swapped my drives and FLASH USB from my old server and plugged them into this one. And Bam. UNRAID started up without a single issue whatsoever. It just said, sweet, more of everything, and went about its business. I was a little worried about how it would handle the Dual 8-Core CPUs (16 Cores Total/32 with Hyper-threading), but it had no issues and that SWEET CPU dashboard is awesome to look at in action. This build maxes out UNRAID's current limit of 30 drives (2 Parity & 28 Data) with a Cache Pool. This is why I decided to get the Storinator Q30 instead of the Q45, or even Q60. Also, the Q45 and Q60 have longer chassis, so the Q30 matched the standard length in a 4U rack-mount. Also, holy cow this thing runs Sooooooo cool. Before, my drives temps would peak at 50C+ (danger zone) during parity/rebuilds in that old Norco 24 bay case. Now they don't break 32C during a parity check. The fan setup in this thing is crazy efficient and the motherboard steps up speeds as temps rise automatically (no fan plugin required). Finally, it is very quite, and lives up to the "Q" in its name. The Q30 is the ultimate sweet spot in my opinion, and pushes UNRAID to its current limits. Note: Serial numbers have been redacted from the pics. Let me know if you have any questions. Enjoy!

@SpencerJ I just setup my Storinator Q30 Turbo with 30 drive bays, and thanks to 14TB drives (the current sweet spot on price per GB) I am now up to 200TB usable, even with Dual 14TB parity drives. 45 Drives makes excellent products and I am very happy with my new Unraid server. My old Norco 24 bay server is going to be used for backups now. I've been an avid UNRAID customer since 2014.