RobJ

If you are running 6.2, it saves the syslog on shut down. Before 6.2, the Powerdown plugin saves the syslog and/or diagnostics.

Pauven, I've posted in the 6.2 Final announcement thread (here) about the dramatic performance boost I got with a change of a single tunable! (By the way, thank you Johnnie!) What's particularly important about it is that users upgrading to 6.2 will have the default md_sync_thresh value of 192. If they have previously raised their tunables, perhaps by using the unraid-tunables-tester, they will probably have terrible performance, as I did. I suspect you are about to see quite a bit more 'clientele' around here. Another thing, I think finding the best relations between the tunables will be just as important as finding the best numbers. They can be part of the guidelines posted for future users.

I just tried it and I got the same result. I set each to XFS and I checked each one. See attached. * At first, I suspected you attached the wrong picture, since it didn't match the current data, perhaps was from earlier. * The diagnostics indicates you probably did a New Config, and have now assigned a Parity drive and Disk 1, and set Disk 1 to XFS. Nothing else is assigned, including the Cache drive. The disk.cfg file, probably from earlier and doesn't have the changes saved into it yet, shows only Disk 1 set to XFS, and the rest of the drives set to Auto, including the Cache drive. The super.dat is empty, no drives assigned at all (normal after a New Config, array not yet started). Now there are intermediate steps where what's on disk is not in sync with what is actual, but once you start the array, it's all saved. Your picture shows the array started, so does not match the data. * One reminder, which I'm sure you know and did, after changing the file system type to XFS you MUST click the Apply button, each time. * Timezone is different than an earlier syslog, probably back to its default, so I assume you may have started fresh, and not set it yet. It's a different machine, but half the drives are the same. * Networking is odd, the network.cfg file is empty. There is some configuration for it elsewhere, so it too may be in an intermediate state, not saved yet. * I noticed that your diagnostics have been downloaded 48 times! That's about double recent numbers of unusual downloads. Sometimes I wonder if there's a computer science class or 2 where the instructor has made the students download these for study. If so, Howdy! to all of you! More likely, I just said Howdy to some silly spambots. We're getting popular with them. * Your flash drive shows evidence of past file system corruption, apparently fixed. Probably fixed by the built-in fsck, but you've also had it in a Windows machine. * OK I'm now examining your syslog, I see you started with an empty super.dat, no assignments, then assigned the drives according to the attached picture, set them to XFS, started the array, but when the system tried to mount each data drive as an XFS partition, all of the formats were wrong, incorrect for unRAID. You then stopped the array, and did another New Config, assigned Parity and Disk 1 and that's where the syslog ends. Unassigned Devices did have a problem like that before, but I thought it had been fixed. Formatting a drive requires a lot of parameters, but all we usually do is 'format drive', not realizing that we are actually choosing a lot of parameters at default values. Clearly Unassigned Devices and unRAID are *not* choosing the same parameter values. While this has come up before, it has probably not been stressed enough that any drive that will be used in an unRAID array should only be formatted by unRAID itself. But all is not lost. You can add drives you know are empty to unRAID, then mount the other drives through Unassigned Devices, copy the data over to unRAID, then add each of those drives to unRAID, where they will be reformatted. To do it quicker, no waiting for clearing and faster writing, you can skip adding the parity drive until the end. You do have CrashPlan backups.

I have to be honest and say this really seems like a significant defect with blake2, when each variant produces a different hash. (Put another way, I think they made a 'hash' of it!) This makes portability very problematic. The only time you are sure it's producing the same hashes is if you use the same tool configured the same way and only on the same machine. But a common usage is to transfer large sets of files elsewhere, and check for corruption at the destination. You won't be assured of the same hashes until you test the other tool, and attempt to configure it the same. Since they seem to be so flippant with the hash produced on the *same* machine, how can you know that a different machine and OS and different CPU maker (Intel vs AMD) will produce the same hash, even if the same blake2 variant is used? It would have to be tested, thoroughly. A little more speed means nothing if guaranteed portability is needed and your chosen blake2 variant can't be trusted.

Usually that happens when you do a New Config. The file system type is set to Auto, which does not appear to be working correctly, not identifying the correct file system on each drive. With the array stopped, just click on each drive and change the file system to XFS, and they should all be good again! Make sure they are ALL correct, before restarting the array. I don't trust the Auto setting any more.

Did you read the post just 3 back of yours? It seemed very clear and comprehensive to me.

I'm not absolutely positive, but it *looks* correct! You may have to adjust your share inclusions (e.g. moving Photos from Disk 2 to Disk 1).

I've never touched the Script button, don't actually know what happens when you click it. I think you're saying there is automatically a default script attached, and somehow through that button I should delete it? (Being lazy here, I really should reinstall UD and find out for myself!)

A couple of observations - Unassigned Devices is actively spinning unused drives up. I recently removed a small drive from the array, and it then showed up in both Unassigned Devices sections. Because it was just sitting there, getting hotter even though doing nothing, I wanted it spun down, until I decided what to do with it. But I discovered that any refresh of the page would spin the drive back up, no matter how I spun it down. Worse, if I used the built in Unassigned Devices to spin it down, it would rather quickly bounce back up. First the spin indicator would change and the temp would disappear, then in less than a minute the spin indicator would change back then the temp would reappear. Only when I manually spun it down at the command line (hdparm -y) did it stay down, at least until the next Main page refresh. I couldn't find any other mechanism involved, so uninstalled this UD, and then found that I could spin the drive down through the built in UD, and it stayed spun down, even with full page refresh. As best as I can tell, UD accesses the drive at start and any Main page change, causing drive spin up. And even though the built in UD doesn't refresh the full page when I click the spin indicator, I suspect it somehow triggers UD to recheck devices, thereby spinning them back up. Possibly that drive check or access is a SMART check or temp check? It doesn't show temps though. Also, clicking the 'Complete' indicator is persistent on the normal Main screen, but does not appear to be in the unRAID boot GUI. I have to turn it off each boot. I don't know if it's a cookie issue in the built in Firefox, or something else. Since it's loaded in RAM, may have to find a different way to save the setting.

I understand this is a tool for more advanced users, but in the current ransomware climate, wouldn't it be better to reverse the model here to be basically read-only, then only allow write access in limited windows (selected users, selected folders)? Seems like that would be safer, especially in any proftpd.conf files provided for new users. Might be a better model for all of us, to always think that way now (read only unless absolutely necessary, then limited)!

Maybe: "Crossflashing Controllers" In fact I have no preference. I figure it should be self-explanatory, simple and easy to find. If you can import those posts, it would be a good base to start from. We should link only from the specific controllers in the compatibility page that are covered on the new wiki pages. There could be a "general" section, where the intention/basics of flashing are explained more in detail, that can be linked to, as a guide for users who have to flash new, unknown hardware or as background information. Well I've gotten it started, and there's lots of info imported, and a skeleton of a page fleshed out, but there's a LOT more work to do! Have at it! It's all yours! I'm afraid much of it was painstaking cut and paste, line by line because each needed different formatting, so it's possible I've made mistakes. I hope not, but someone will need to double check all info. There's important info missing too, like how do you flash! And I didn't know how best to set up the bottom sections, like boot disks, tools, etc. Let me know if there's anything else you would like me to do. The page is at -> Crossflashing Controllers

Check the SMART report, it will tell you about the drive, any issues with it. Then test the drive for Preclear signature (although I don't recall if the plugin has that test). Those 2 tests are probably the most important, when a Preclear finishes. About the only other thing tested is whether every bit is zero, but that only very rarely fails, and would indicate something really wrong with the drive, and you'll see that soon enough. Very rare though.

I suppose if I wanted to enforce this, in a way that is fair and avoids squabbles, I'd create a policy of BYOD, Bring Your Own Disk! Then each one uses and writes *only* to their disk. If they want more space, they buy a bigger disk, and replace theirs. No one ever has to deal with anyone else's greed or clutter. This doesn't preclude having other shared spaces, if you choose. To avoid fights, it's always best for each shared space to have only one owner, one 'ruler'. Their space, their rules, even if shared access.

What is "Boot GUI mode", and how do I change to it? Some users run unRAID as headless, and if you do, then this FAQ is not for you. The Boot GUI is a GUI replacement for the text based terminal screen on the server console that we have always seen after boot, until now. It provides the same unRAID screens on the server monitor that you normally get when you open your unRAID server from a browser on another desktop. One important requirement - a mouse! If you have never viewed a GUI on your server machine, then you may not have one attached, and any GUI requires a mouse. Because support is built in, getting a mouse working should be easy, but may not be (and you may have to reboot). To set the GUI mode, go to the Main screen and click on your unRAID flash disk (on the word 'Flash'), to open its settings page. Go down to the Syslinux Configuration section. This is a viewer and editor of your syslinux.cfg file, the file that controls the boot menu for your server. After some preliminary general settings lines, you will see a series of sections that begin with label. Each section controls a separate menu line on your boot menu. In one of them is the line "menu default", which marks the menu item that will be performed if nothing is selected within the allotted time. To change the default menu selection, simply move the "menu default" line into the section you want to be the default, then click Apply then Done. Try it and see, you can always change it back. One warning! This uses Firefox as the browser to open the unRAID server management pages. It's fine for that, but because it's a browser, it will be tempting to go on the Internet. It is NOT configured to securely access the Internet! Don't do it! To be safe, use it only for managing your unRAID server. (This author is not sure of the security implications and dangers here, and prefers to err on the side of safety.) An example of a modified syslinux.cfg - default /syslinux/menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label unRAID OS kernel /bzimage append initrd=/bzroot vga=6 label unRAID OS GUI Mode menu default kernel /bzimage append initrd=/bzroot,/bzroot-gui vga=6 label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode vga=6 label Memtest86+ kernel /memtest

Yes, I'm in. Would you please prepare the backbone somewhere in the wiki? Then I will work on the content. Excellent! Some opening questions - * What would you like for the page name? some ideas - Flashing LSI Controllers, Controllers and Firmware, Flashing SATA Controllers, etc * I can link to it from the Hardware Compatibility page, and I can modify the first post of this thread to point to it. Anywhere else you would like? * Would you like me to import Madburg's first 4 posts and reformat for wiki syntax? Or leave it all to you. You would still have to update them. You also may have your own ideas for formatting style.

Fireball, it really seems way past time to put this all in a wiki page! Any interest? Then we could move madburg's first 4 posts over to it, and you could update them all for current versions, plus you could add other sections that are relevant, such as about UEFI, etc. It would be great to extract all the nuggets hidden throughout this long thread, into one place! A place that anyone can update (but is patrolled). I suspect it might even make your life easier, less support necessary, because good info keeps getting buried on back pages. I'll be happy to assist any way you like, just tell me what you want.

There are a couple of additional scripts for the user scripts plugin dealing with turbowrite. Simple enable / disable, and enabling / disabling based upon the number of drives spinning at the time (although this one really would require a spindown time of 1 hour to be set, since the minimum frequency for script running is every hour). Almost as easy as bonienl creating a plugin. I'm sorry, I completely forgot about them, which is nuts since I've added them and been on the User Scripts screen rather often in the last few days! I do still think a Main screen button/toggle would be somewhat quicker, more convenient. Unless you add an option for a Main screen button for User Scripts!?!

Also check out the User Scripts plugin, which now includes scheduling.

dlandon has now added disabling of Telnet and FTP to the Tips and Tweaks plugin. That makes it by far the easiest way to do it. I've added the plugin link to the tips on the Tips and Tweaks wiki page. I just added a performance tip (turning on turbo write) to the page, as it provides such a great write speed boost. I did make sure to mention the downside. I think it would be great to have a toggle for turbo write on the Main page. Plugin anyone? ken-ji, I haven't gotten to your tip yet, as I wanted to understand it better, as soon as I have time. I'm afraid the only ELK's I knew about were 4 legged!

Changing the unRAID port should really be in a FAQ or wiki somewhere, or perhaps a built-in setting. I believe this still works the same in v6.

Nice! One suggestion, if disable, enable turbo write before starting, clearing can be up to 3 times faster. It's such a major improvement that you're right, it really should be on. I do include instructions on the Shrink array wiki page to turn it on (and I'm going to add it to the Tips and Tweaks wiki page), but I think I should find a programmatic way to add it to the script. Perhaps capture their current setting, turn it on, then restore their setting once the script completes. Ooooh nice! The man page I checked didn't have that. I should have checked its own '--help'. Now to see if I can get it to display correctly and usefully in the User Scripts display box... That may not be easy. Be great for standalone script use though. Update: added the progress, works great! Thanks again!

Well, I've created my first script for this, and overall the experience has been both nice - and frustrating. The frustrating part is mainly fighting bash syntax! I've created the Clear an array drive script, for use in removing drives from unRAID without losing parity. I created it first to run standalone, then adapted it to work within User Scripts. Some comments - * Really needed a way to interact with the user, but developed ideas, kludges to work around that lack. I suppose some might say I should have kept it for standalone use only, where I could possibly interact more, but I wanted this to be useful for new users, with no command line usage necessary. Perhaps some day, you'll figure out ways to add a bit of interactivity? I don't know enough to help, yet. * I was a little disappointed to discover that the script display box won't display a line until you terminate it, with a newline. I like to keep the display clean, fewer lines, and had hoped to display a bit of progress info on the same line, using \r to overwrite the current line. But it won't display until you go to the next line. I doubt there's anything you can do about that, so not your fault. I've experienced this in other 'venues' too. It would have been nice for the clearing progress though, where I could keep updating the same spot with the percentage complete. * This script necessarily runs in the foreground. I suppose it could run in the background, but not sure it should. I suspect there are some scripts that are always best in one or the other but not both. A low priority suggestion - utilize user.scripts variable(s) in the early comments. For example, "# user.scripts.foreground=off", which would mean only the 'Run in background' button would appear. For my script, I would put "# user.scripts.background=off", which would mean no 'Run in background' button. * Running in the foreground of course, it hangs the browser tab, for the entire time it's clearing, which is hours! No help for that, I do need to show some stuff to the user, and allow them to either proceed or cancel. I don't think there's any way to do that in the background. * In the script display box, are bold or colors possible? It doesn't accept bash shell escaped color codes. * Might be nice to add a "Add user script" button, which would check the root of the flash for a zip file with a folder that had 'description' and 'script' in it, and installed the folder in the right place. It would make it easier for less-technical users (like those that might use my script) to install a script (just download the one they want to the flash and click this button). The tool zipinfo is built-in, and using zipinfo -1 - it gives you a simple and clean string blob of the zip contents. But I'm sure you have lots of other things to work on!

Clear an unRAID array data drive (for the Shrink array wiki page) Mod note: this script usually takes a much longer than normal time to clear a drive with newer Unraid releases, recommend using the "Remove Drives Then Rebuild Parity" Method or if you really want to clear the disk do it manually. This script is for use in clearing a drive that you want to remove from the array, while maintaining parity protection. I've added a set of instructions within the Shrink array wiki page for it. It is designed to be as safe as possible, and will not run unless specific conditions are met - - The drive must be a data drive that is a part of an unRAID array - It must be a good drive, mounted in the array, capable of every sector being zeroed (no bad sectors) - The drive must be completely empty, no data at all left on it. This is tested for! - The drive should have a single root folder named clear-me - exactly 8 characters, 7 lowercase and 1 hyphen. This is tested for! Because the User.Scripts plugin does not allow interactivity (yet!), some kludges had to be used, one being the clear-me folder, and the other being a 60 second wait before execution to allow the user to abort. I actually like the clear-me kludge, because it means the user cannot possibly make a mistake and lose data. The user *has* to empty the drive first, then add this odd folder. #!/bin/bash # A script to clear an unRAID array drive. It first checks the drive is completely empty, # except for a marker indicating that the user desires to clear the drive. The marker is # that the drive is completely empty except for a single folder named 'clear-me'. # # Array must be started, and drive mounted. There's no other way to verify it's empty. # Without knowing which file system it's formatted with, I can't mount it. # # Quick way to prep drive: format with ReiserFS, then add 'clear-me' folder. # # 1.0 first draft # 1.1 add logging, improve comments # 1.2 adapt for User.Scripts, extend wait to 60 seconds # 1.3 add progress display; confirm by key (no wait) if standalone; fix logger # 1.4 only add progress display if unRAID version >= 6.2 version="1.4" marker="clear-me" found=0 wait=60 p=${0%%$P} # dirname of program p=${p:0:18} q="/tmp/user.scripts/" echo -e "*** Clear an unRAID array data drive *** v$version\n" # Check if array is started ls /mnt/disk[1-9]* 1>/dev/null 2>/dev/null if [ $? -ne 0 ] then echo "ERROR: Array must be started before using this script" exit fi # Look for array drive to clear n=0 echo -n "Checking all array data drives (may need to spin them up) ... " if [ "$p" == "$q" ] # running in User.Scripts then echo -e "\n" c="<font color=blue>" c0="</font>" else #set color teal c="\x1b[36;01m" c0="\x1b[39;49;00m" fi for d in /mnt/disk[1-9]* do x=`ls -A $d` z=`du -s $d` y=${z:0:1} # echo -e "d:"$d "x:"${x:0:20} "y:"$y "z:"$z # the test for marker and emptiness if [ "$x" == "$marker" -a "$y" == "0" ] then found=1 break fi let n=n+1 done #echo -e "found:"$found "d:"$d "marker:"$marker "z:"$z "n:"$n # No drives found to clear if [ $found == "0" ] then echo -e "\rChecked $n drives, did not find an empty drive ready and marked for clearing!\n" echo "To use this script, the drive must be completely empty first, no files" echo "or folders left on it. Then a single folder should be created on it" echo "with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen." echo "This script is only for clearing unRAID data drives, in preparation for" echo "removing them from the array. It does not add a Preclear signature." exit fi # check unRAID version v1=`cat /etc/unraid-version` # v1 is 'version="6.2.0-rc5"' (fixme if 6.10.* happens) v2="${v1:9:1}${v1:11:1}" if [[ $v2 -ge 62 ]] then v=" status=progress" else v="" fi #echo -e "v1=$v1 v2=$v2 v=$v\n" # First, warn about the clearing, and give them a chance to abort echo -e "\rFound a marked and empty drive to clear: $c Disk ${d:9} $c0 ( $d ) " echo -e "* Disk ${d:9} will be unmounted first." echo "* Then zeroes will be written to the entire drive." echo "* Parity will be preserved throughout." echo "* Clearing while updating Parity takes a VERY long time!" echo "* The progress of the clearing will not be visible until it's done!" echo "* When complete, Disk ${d:9} will be ready for removal from array." echo -e "* Commands to be executed:\n***** $c umount $d $c0\n***** $c dd bs=1M if=/dev/zero of=/dev/md${d:9} $v $c0\n" if [ "$p" == "$q" ] # running in User.Scripts then echo -e "You have $wait seconds to cancel this script (click the red X, top right)\n" sleep $wait else echo -n "Press ! to proceed. Any other key aborts, with no changes made. " ch="" read -n 1 ch echo -e -n "\r \r" if [ "$ch" != "!" ]; then exit fi fi # Perform the clearing logger -tclear_array_drive "Clear an unRAID array data drive v$version" echo -e "\rUnmounting Disk ${d:9} ..." logger -tclear_array_drive "Unmounting Disk ${d:9} (command: umount $d ) ..." umount $d echo -e "Clearing Disk ${d:9} ..." logger -tclear_array_drive "Clearing Disk ${d:9} (command: dd bs=1M if=/dev/zero of=/dev/md${d:9} $v ) ..." dd bs=1M if=/dev/zero of=/dev/md${d:9} $v #logger -tclear_array_drive "Clearing Disk ${d:9} (command: dd bs=1M if=/dev/zero of=/dev/md${d:9} status=progress count=1000 seek=1000 ) ..." #dd bs=1M if=/dev/zero of=/dev/md${d:9} status=progress count=1000 seek=1000 # Done logger -tclear_array_drive "Clearing Disk ${d:9} is complete" echo -e "\nA message saying \"error writing ... no space left\" is expected, NOT an error.\n" echo -e "Unless errors appeared, the drive is now cleared!" echo -e "Because the drive is now unmountable, the array should be stopped," echo -e "and the drive removed (or reformatted)." exit The attached zip is 'clear an array drive.zip', containing both the User.Scripts folder and files, but also the script named clear_array_drive (same script) for standalone use. Either extract the files for User.Scripts, or extract clear_array_drive into the root of the flash, and run it from there. Also attached is 'clear an array drive (test only).zip', for playing with this, testing it. It contains exactly the same scripts, but writing is turned off, so no changes at all will happen. It is designed for those afraid of clearing the wrong thing, or not trusting these scripts yet. You can try it in various conditions, and see what happens, and it will pretend to do the work, but no changes at all will be made. I do welcome examination by bash shell script experts, to ensure I made no mistakes. It's passed my own testing, but I'm not an expert. Rather, a very frustrated bash user, who lost many hours with the picky syntax! I really don't understand why people like type-less languages! It only *looks* easier. After a while, you'll be frustrated with the 60 second wait (when run in User Scripts). I did have it at 30 seconds, but decided 60 was better for new users, for now. I'll add interactivity later, for standalone command line use. It also really needs a way to provide progress info while it's clearing. I have ideas for that. The included 'clear_array_drive' script can now be run at the command line within any unRAID v6, and possibly unRAID v5, but is not tested there. (Procedures for removing a drive are different in v5.) Progress display is only available in 6.2 or later. In 6.1 or earlier, it's done when it's done. Update 1.3 - add display of progress; confirm by key '!' (no wait) if standalone; fix logger; add a bit of color Really appreciate the tip on 'status=progress', looks pretty good. Lots of numbers presented, the ones of interest are the second and the last. Update 1.4 - make progress display conditional for 6.2 or later; hopefully now, the script can be run in any v6, possibly v5 clear_an_array_drive.zip clear_an_array_drive_test_only.zip

I too have been enjoying this tool, has helped with some testing I've been doing. Thanks! We're getting off-topic though, perhaps it deserves its own thread?

Thanks for the new feature to disable FTP and Telnet! Nice work! And quick! I do think there will be users that want FTP but not Telnet, or want Telnet but not FTP. Might be better to have separate toggles for them.