Leaderboard

Just a simple little plugin to act as a front end for any little scripts that you may have that you may need to run every once in a while, and don't feel like dropping down to the command line to do it. (Or anything that I happen to run across here on the forum that will be of use to some people) Install it via Community Applications Only a couple included scripts: - Delete .DS_Store files from the array - Delete any dangling images from the docker.img file - Display the size of the docker container's log files (to see if a docker app is filling up the image file through excessive logging) Additional Scripts from myself (and hopefully other users) can be found here: To add your own scripts: Within the flash drive folder config/plugins/user.scripts/scripts create a new folder (each script is going to have its own folder) - The name doesn't matter but it can only contain the following characters: letters ([A-Za-z]), digits ([0-9]), hyphens ("-"), underscores ("_"), colons (":"), periods ("."), and spaces (" ") Or, you can hit the button that says "Add Script", then give the script a name. Hovering over the script's name will then give you additional options. Including online editing... Create a file called description that contains the description of the script. Create a file called script this will be the actual script. Few notes: So as to make things easier for people: The script file can be created with any text editor you choose. DOS line endings will be automatically converted to Linux style endings prior to execution. #!bin/bash is automatically added to every script prior to execution to help out the noobies EDIT: This is only added if no interpreter is specified (ie: #!/bin/bash) If an interpreter is already specified (ie: #!/usr/bin/php), then line is not added Techie Notes: The scripts are actually copied and executed from /usr/local/emhttp/plugins/user.scripts/ /tmp/user.scripts/tmpScripts so if there are dependencies (other scripts, etc) stored in the same folder as the script file, you will need to specify the full path to them. Interactive (ie: answering yes/no to a question) scripts will not work.

Hi, I have a btrfs cache pool of 4 ssd drives, which hosts the docker image as well as downloaded data. I noticed that some of the docker apps were occasionally having issues like "database locked" and "write error, disk full?", etc. After thorough testing, I realized that whenever there is a large file transfer on the cache pool, where the file is read from and written to the cache drive, the server temporarily locks up. The unraid gui is unreachable, the docker apps stall and their guis are unreachable and ssh access is slow and occasionally hangs in the middle of a basic operation like "pf -es". This continues until the file transfer is over, and a few minutes later, everything is back to normal. This seems to happen with files larger than about 10GB. A typical scenario is, sabnzbd downloads a file over 10GB, during unrar of the file to the cache pool, everything else is locked up. Then, when the file is being moved from the sab temp folder to the Movies folder on the cache pool by radarr, again, everything else is locked up. Because of the lock up, it is difficult to trouble shoot. I don't know what else to try or test. I have sabnzbd only using certain cores and not all, so the issue should not be due to high cpu utilization during unrar. Therefore I believe it is due to disk io that is fully taken up by the file transfer. Attached is my diagnostics, which should include info from this morning's 20GB file transfer (unrar operation failed a couple of times due to disk full message although there was 100GB of free space, but then succeeded on the third try). I would appreciate any ideas or suggestions. Thanks PS. File transfers from the cache pool to the array are completely fine. Mover does not affect general server operations and neither does a regular copy to the array. tower-diagnostics-20170627-1153.zip

I don't believe there is log rotation enabled in unraid's implementation of docker. There doesn't seem to be a limit on log size either. I kept running out of space in my docker image and finally realized that a few of the containers were filling up the image with their logs. One container had a log that was 2.8GB (couchpotato) The temporary solution was to reinstall the container and it reset the files in /var/lib/docker/containers and got rid of the large logs. It freed up a ton of space, too. (For any users interested, the easiest way to reinstall is to click on the container image in the unraid gui, select edit, don't change any settings and just hit save, it will reinstall the container with the same settings) Is there anyway a limit on size can be implemented, or log rotation? Or perhaps an option to move the logs somewhere else where there is more storage available? Thanks PS. If anyone's interested in checking how big their logs are, type this in the unraid terminal and it will list the largest logs: du -ah /var/lib/docker/containers/ | grep -v "/$" | sort -rh | head -60

I would love to be able to view the cpu thread pairs from the vm templates like this

v3.0.0 is now available ! 2017-06-28 - 3.0.0 This is a major internal redesign to allow for new features to be added. Starting with this release, you can now COPY files (as opposed to only MOVE them in the previous version). In upcoming releases, you will be able to validate the copy (VALIDATE operation) and use the completely new GATHER operation. GATHER consolidates user share folders into a single disk (think for example when you want to consolidate all the season of a tv show into one disk) Additionally, a support fund was established, if you want to show your support to the plugin developer. These are other changes: - Prevent spurious rsync lines in logs - Use system font in UI - Throttle websocket updates (for dry-run simulations) - Improve hostname detection in plugin's unRAID settings page - Bug fixes and improvements

Wait for the next rc...

Ok, looks like I will need to add a parameter for the output format...!

Here is mine for an example. Look at my Anime one. You have to remember that it depends on where you map your folders in the Edit Configuration where its going to show up in Plex. For instance if you map everything to /Media/Anime it would show up in Plex under the /Media folder. So its up to you where you want things. I was lazy and just mapped everything to / which shows up as /Anime

OK, I have looked at your diagnostics now and you have a bad disk: Serial Number: WD-WCC4E2ZZFYAH 5 Reallocated_Sector_Ct 0x0033 133 133 140 Pre-fail Always FAILING_NOW 1963 9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 24 Looks pretty new too. You should replace it with another disk and return it. Now would be a good time to learn. Preclearing is the way most people test a disk before using it. Every bit of every disk must be good because every bit of every disk is required to rebuild a failed disk.

I haven't looked at your diagnostics yet. So is it rebuilding now? What do you mean by slow? It will take a considerable amount of time to rebuild a 4TB disk, several hours, because it has to write every bit of the 4TB, regardless of how many files may have been saved. Parity doesn't know anything about files. It should show you the speed of the rebuild in Main - Array Operations and give you an estimated completion. Post a screenshot if you want.

Hi Love this docker and the Latest Update Is there any chance it could be change to respect directories ? For example i have my watch folder set up like this /watch/TV Shows /watch/Movies However when the files are converted they are just put straight in to the output folder ignoring if its a tv show or a movie and therefore Sonarr and radarr don't know the conversion has finished i feel like this would be a useful change

So far it doesn't sound like you have done anything really bad. Just don't do this: That doesn't really make any sense anyway because clearing and formatting are not even remotely similar, and neither would help. Go to Tools - Diagnostics and post the complete diagnostics zip.

@Frank1940 Are you referring to the Ransomware Protection plugin? To beef up my post (and to intrude into this conversation) -- my current strategy has been to limit access to shares hosted on all of my servers down to a single management system (you could potentially have this under vlan, with no internet access). As for accessing my data -- well, I'm leaving that up to the applications that need access (I'm also limiting write access), e.g; My movie collection can only be accessed by end-users via Plex. If a particular piece of software needs to manipulate files, then that software is usually ran via a container (directly on the host system, so no need for a share -- side benefit here is version control). If I need to have two servers talking, then this is done over NFS -- I lock this down to their individual IPs as well as a dedicated account for their own use. This is not as secure though if the server itself is compromised, let's say a Windows Server (so I simply don't have my Windows Server setup to talk over NFS). What of ransomware targeting Linux-based systems? Simple really: limited access. Segregated services (Docker is great for this -- same can be said about VMs). While we lose the convenience of easily accessible shares, there should be no need to go full tinhat.

be an optional add on to @Squid's 'docker safe newperms' FTFY

One thing I've done is utilize FIND command to make files read-only if they're over a certain size on my media shares. This leaves the files produced by media managers to be updated, such as .nfo, .xml, .txt or even the backgrounds and logos, since those are all easily replaceable. To genuinely make them secure I suppose they should also be chown'ed to a different user too. This could replace the 'newperms' script.

I will forward this to the rest of the team for feedback. Thanks for posting and including all the details!

Hallo I got my Ryzen a week ago. thanks for all the good reading. I just got the IMMUO a lot better by using "pcie_acs_override=downstream,multifunction" IOMMU group 0 [1022:1452] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 1 [1022:1453] 00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 IOMMU group 2 [1022:1452] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 3 [1022:1452] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 4 [1022:1453] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 IOMMU group 5 [1022:1453] 00:03.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 IOMMU group 6 [1022:1452] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 7 [1022:1452] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 8 [1022:1454] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1454 IOMMU group 9 [1022:1452] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 10 [1022:1454] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1454 IOMMU group 11 [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 12 [1022:1460] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1460 [1022:1461] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1461 [1022:1462] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1462 [1022:1463] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1463 [1022:1464] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1464 [1022:1465] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1465 [1022:1466] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1466 [1022:1467] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1467 IOMMU group 13 [1022:43b9] 03:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 43b9 (rev 02) IOMMU group 14 [1022:43b5] 03:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] Device 43b5 (rev 02) IOMMU group 15 [1022:43b0] 03:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b0 (rev 02) IOMMU group 16 [1022:43b4] 1d:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) IOMMU group 17 [1022:43b4] 1d:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) IOMMU group 18 [1022:43b4] 1d:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) IOMMU group 19 [1022:43b4] 1d:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) IOMMU group 20 [1022:43b4] 1d:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) IOMMU group 21 [1022:43b4] 1d:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) IOMMU group 22 [10de:1381] 21:00.0 VGA compatible controller: NVIDIA Corporation GM107 [GeForce GTX 750] (rev a2) IOMMU group 23 [10de:0fbc] 21:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1) IOMMU group 24 [1b4b:9230] 23:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller (rev 11) IOMMU group 25 [1b4b:9230] 24:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller (rev 11) IOMMU group 26 [1b21:1343] 25:00.0 USB controller: ASMedia Technology Inc. Device 1343 IOMMU group 27 [8086:1539] 26:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 28 [10ec:8168] 27:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06) IOMMU group 29 [10de:040e] 28:00.0 VGA compatible controller: NVIDIA Corporation G84GL [Quadro FX 570] (rev a1) IOMMU group 30 [10de:128b] 29:00.0 VGA compatible controller: NVIDIA Corporation GK208 [GeForce GT 710B] (rev a1) IOMMU group 31 [10de:0e0f] 29:00.1 Audio device: NVIDIA Corporation GK208 HDMI/DP Audio Controller (rev a1) IOMMU group 32 [1022:145a] 2a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a IOMMU group 33 [1022:1456] 2a:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Device 1456 IOMMU group 34 [1022:145c] 2a:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 145c IOMMU group 35 [1022:1455] 2b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455 IOMMU group 36 [1022:7901] 2b:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 37 [1022:1457] 2b:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Device 1457 3 GPU and 3 vm running. and soundcard working in vm Card Asus Prime X350 Pro Unraid 6.3.5

OP usually means original poster (person who started a thread). But I think this is referring to the original post, the first post in this thread.

Just a further comment. This must be an edge case of enormous unlikelyhood. Think of the number of computers based on those chips, and the millions upon millions of hours of testing and use. If a specific user were to randomly hit this issue once, it would be unlikely. But to hit it repeatedly within hours of booting - if that were the kind of symptom, it would have been found and fixed long ago IMO. I remember in college there was an assignment to implement a very very basic multi-process "OS". This was on a now ancient Z80 processor. The trick was to interrupt a process (easy to do), and then precisely remember the processor state. Since each thing you do changes the processor state, this was a bit tricky to not destroy the state of the registers as you stored things away. And then to put things back precisely so that the interrupted process was oblivious to the fact out was interrupted. We were running like 10 processes in parallel giving each a time slice to run. The running processes were all the same - a mathematical calculation of some kind. It took probably 20 seconds, displayed the same answer, and looped to do the same thing over and over. I remember it being fun, because you could run several iterations and it would work and then you'd get a weird answer being displayed and you'd have to figure out what you were not preserving. After several test runs I got it ruining consistently, and was letting it run and run, several minutes - perfect. I was the first one in the lab to get it running, and was chatting with the TA when I got a wrong answer displayed. Crap. It had run a long time. Went through the code line by line, with the TA, and it was perfect. Others started finishing and ran theirs for several minutes, but left on for 15 minutes or more, it would have one wrong answer. TA couldn't explain it, and we went home confused. Next class teacher explained what happened. There is a command DAA (decimal adjust accumulator if memory serves) that is dependent on an invisible carry flag that the prior instruction could set. He then explained a method to preserve even that state that aforeto we had no idea existed. So maintaining state is tricky, and even is your think every nuance is covered, it may not be. I have to think this issue is similar to my college experience (in infinitely more complexity) of some extremely nuanced situation whet state is not properly preserved.

http://www.insanelymac.com/forum/topic/309087-insanely-fast-virtual-mac-qemu-ovmf-clover-and-native-graphics/page-25 Looks like this requries QEMU 2.9.0 to work. The parameters are simple enough to add to the unraid XML, but I haven't updated to 6.4-rc yet. Once 6.4 is stable and those parameters are confirmed working we should be able to use macOS on unraid with unpatched clover versions.

Teaser ... this is how it looks for me

I would like to move beyond talking about Wannacary as it is too narrow a focus. Happy to fork the thread although all the right people are monitoring this one so if we can stay it would be better. There are several protocol level problems with SMBv1 that cant and wont ever be fixed and it has been deprecated as a recommended protocol for quite some time now by all the relevant players. A generic non exploit related example of SMBv1 failure is that there are known issues where legacy devices/OS will "fail low" and handshake below SMBv2 even when they are far more capable just because SMBv1 exists on the remote server. I do not deny that universally disabling SMBv1 would break equipment and it is why i specifically did not suggest this but it is clear that there is a subsection of the userbase that do not need it, some that dont need it but unknowingly still use it and yet others who have no idea and will need some help with the topic as a whole. Saying all that I can see that more compatibility is a simpler stance and has many benefits in itself but I definitely dont believe it is the most secure stance we could take. Ultimately the decision is yours if we want to take a proactive approach to security vs ultimate compatibility.

Will do once we debate it out here first.... unless I nailed it on first try? Also http://www.theregister.co.uk/2017/06/22/latest_windows_10_build_kills_exploited_smb1/

I have put together a tutorial showing how to link media containers through Binhex's excellent vpn containers.

A couple of things to take this a step or two further... Auto 'tick' the cpu thread pair when you choose CPUs NUMA node management so memory is assigned from the same NUMA node as the selected CPU(s) (Dual CPU system problems!) Building on the second point... a flag if you're passing hardware which isnt on PCIe lanes attached to the CPUs?.. not sure if thats even possible though?

A good starting point would be from @NAS s post in the Accelerator Drives thread. I would start with a "find" command and piped to exec. The -ls parameter does a file listing of the results. So to find all the files larger than 10 Megs in size and give a listing: find /mnt/user/Media/ -type f -size +10M -ls To find files smaller than 10 Megs in size and give a listing: find /mnt/user/Media/ -type f -size -10M -ls Other units available for size is 'k' for Kilobytes, 'c' for bytes, 'M' for Megabytes, and 'G' for Gigabytes. To change permissions to READ only for files larger than 10 Megs in size: find /mnt/user/Media/ -type f -size +10M -exec chmod 444 {} \; Then to change owner to ROOT for files larger than 10 Megs in size: find /mnt/user/Media/ -type f -size +10M -exec chown root {} \;

I think you're running a risk by scheduling correcting checks - suppose a disk fails, or simply drops off-line, during a correcting check. It's far safer to schedule a non-correcting check. Most of the time the result will be no errors found. On the very odd occasion an error is found then yes, you need to run a correcting check - but on your own terms and after taking whatever remedial action you see fit first.

The script file also supports some inline variables which will customize the GUI for the script in question description This is the description of the script - ie: what will show up on the UI for the plugin foregroundOnly Setting this to be true disallows background running (and scheduling) of the script backgroundOnly Setting this to be true disallows foreground running of the script arrayStarted Setting this to be true will only run the script (foreground or background or scheduled) if the array is up and running name this is the name of the script. Without this variable, the GUI will display the folder's name argumentDescription if present this will bring up a pop up asking the user for the argument list for the script. Note that currently arguments do not accept spaces contained within one argument (ie: quoting and escaping spaces does NOT work) argumentDefault this is the default arguments for the above clearLog Set to be true to have the log deleted prior to execution of the script (ie: logs are only for the last execution of the script) noParity Set to be try if the script is not allowed to run when a parity check / rebuild is in progress How to implement these variables: Immediately after the interpreter line (eg: immediately after the #!/bin/bash line), add these lines if you choose (you do not need to add all of them if you don't require them) #description=this is the description of the script #foregroundOnly=true #backgroundOnly=true #arrayStarted=true #name=blah blah blah #clearLog=true #argumentDescription=This is the description of the argument(s) #argumentDefault=defaultArg1 defaultArg2 etc #noParity=true After the first non comment line within any script, parsing for these variables stops. (IE: they have to be right at the top of the script file) Note that you do NOT have to have any or all of the lines contained within any particular script. (Also, if you do have the description file present, then its contents takes precedence over the description variable) *PHP scripters: You can also place these variable lines immediately after the <? or <?PHP line