Addy90

Yes, definitely. I believe Limetech has loads of automated test patterns for all edge cases they know to be sure about changes that can cause data loss. Of course this is nothing one can implement quickly, but I think as it is possible to pause and continue a parity check already during operation, it should also be safe after reboot. Especially because a lot of conditions are checked after reboot if the array and all disks are available and there were no changes to the array. But it is not for me to assume too much here, it is just a Feature Request thread and I wanted to ask for something that I would love to see and that I think is not impossible to do (much easier than double parity or multi-streaming and other features we have seen recently). It is just a fair feature request in my opinion, and some suggestions about possible implementations because you made me to write about it. The rest is hope and patience, from which I have both. I prefer a solid tested feature than any other feature, like most of us here do; this is one reason for us to use Unraid, because it works. Don't agree?

Ah, yes, of course it is nearly impossible for plugin development! I know you have it on the Wish List for your great Parity Check Tuner plugin! But it is exactly as you describe it there (copy from your Parity Check Tuner page) The offset ability has to be implemented by Limetech! This is why I ask in the Unraid Feature Request thread and not in your plugin thread, because I know that it is not possible for you and that you need that feature. But I think for Limetech it is not very difficult, because this is what I think: There is a function md_do_sync() with a variable mddev->curr_resync in the md daemon that holds the position of the current parity check run. This variable is incremented (mddev->curr_resync += sectors) within a while loop until the maximum sector (mddev->recovery_running) is reached, which marks the end of the parity check operation. Parity check can already resumed by giving options to check_array() function which revives the mddev->recovery_thread that continues with the current mddev->curr_resync position held in memory. I think it is easily possible to read the content of this variable (and others if needed, maybe with a checksum, just to be sure no modification happened). The status_md() function prints out the content of this sync-position variable, for example (seq_printf(seq, "mdResyncPos=%llu\n", mddev->curr_resync/2);). Maybe it is already possible to get the contents of the needed variables with this function. So when you can get the content of this variable, the check_array() function only needs an additional parameter for resuming operation at a specific position. It then sets the mddev->curr_resync, wakes up the sync-thread, and operation continues where it left off after reboot! This parameter can be read from a file, controlled by the checksum and the mdcmd tool can be improved by accepting this value for parity check via command line. Everything should be possible via mdcdm calls via UI, the functions permit the setting of the current position and the UI can save the current value to a file (with checksum just to be sure) and read it back after reboot and only delete the position file in case the operation is running and write it when it was paused or array shut down manually, so that when there is an unclean shutdown or the check ended successfully, the file is not there and next start check if there is a file and if not, starts from the beginning. I think, for Limetech, it is not very difficult to permit this operation. I would LOVE to see it Would made mine and some other days, also @itimpi's days brighter PS: I don't mind pausing and resuming parity sync over multiple reboots. It is as easy as just shut down the array, file with position is saved, and after reboot, parity check tells in UI that the last check was not finished and can be resumed (or ask to start from beginning by checkmark, which calls mdcmd without position argument). Not difficult for the user, but very handy for the use-case! PPS: As parity check is also used for rebuilding disks as far as I understood, it should either be possible to resume rebuild after reboot, but in this case, I would certainly warn about that to do or gray out the option to shut down, but it should be possible technically. In case of unclean shutdown, we have to restart either way.

What I would find really great would be the possibility to pause parity sync over multiple reboots! I have two reasons for this: First is: In my situation, I am living in a flat and normally I have all doors open during night for better air support and because I have no radiator in the sleeping room, so I get my heat from other rooms. Unfortunately, parity sync takes longer than 24h and this is the only time in month I have to close doors because of the noise, and then it gets cold, too. Second reason is that normally I am only syncing backups from my other systems several times a week up to once a day, or I need some older data stored on the array that is not on my live systems some times in month, means that my array is shut down most of the day and only online for some hours, maybe half a day, because of noise and energy costs. I could use that time when I need it sparely, but ready, to continue parity sync during that time. This saves a lot of energy, because the only time during month when it is running for more than 24h, even during night when I don't access it, is for parity sync. So what I would appreciate is that the parity sync operation writes some stat file when paused explicitely and/or clean system shutdown to flash drive and that it gives the possibility to continue where it left off before (or restart from beginning, for example with an additional check-mark). Like this, parity sync could be paused and resumed whenever the system has not to be shut down for whatever reason. I don't think this is difficult because parity sync operation knows always where it is and I know there is parity check tuning plugin that lets run parity sync in intervals, but it does not survive reboot, which is critical for my use case. I hope this can be possible! Thank you! BTW this was already requested, but maybe the importance was not given as much as in my text:

I did not read the whole article, but when you seek for an enterprise solution, use something like CEPH. https://ceph.io/ I am using Unraid in my home setup as it is an incredible flexible and cheap solution for backup data and archiving. Nothing is as flexible and cheap in setup in my opinion. At work, I setup a CEPH cluster, here you can have hundreds of drives on dozens of servers with failover and parity over multiple servers. You can reboot parts of the cluster for updates with no down time and have virtual machines use CEPH as underlying network storage, for example with Proxmox (built-in). If you need additional backups, use a second technology like GlusterFS that also can run over multiple disks on multiple servers. If there is a problem with CEPH, your second storage on GlusterFS is not affected. Never confuse backups with failover! And then you have your scalable, enterprise ready system with multiple parity disks > 2, many data disks > 28, server and even location failover and backup. Nothing of this can be given by Unraid and Unraid should not be designed for this, as it would complicate the setup for every home and small business user - and it would compete with CEPH and GlusterFS, but for no reason as these systems already solve the problems you seem to seek for. CEPH also supports multiple pools with different redundancy settings, different synchronization strategies, distinct SSD and HDD pools on same infrastructure for high performance and high storage pools. You can choose which disks you want in which pool, you can choose the parity calculation, and more... you can use CEPH as block storage and file storage with CephFS. CEPH also has integrity verification built in (scrubbing). And and and... everything enterprise ready storage needs! I am sure it is possible to run SMB sharing over CephFS pools, though I am not sure about SMB failover, so maybe the gateway running SMB might have downtimes on reboot. Maybe SMB can be made high availability via keepalived (VRRP) and conntrackd, then the SMB shares would be failover, too, like the whole storage system. I don't want to make advertising for these technologies. I am not involved in any development of them, but I use them all where applicable. Unraid at home, CEPH for high performance, high availability, high scalable enterprise VM/Container storage at work, GlusterFS for backups of CEPH VMs/Containers at work. keepalived for failover of some services (but not SMB), conntrackd on VyOS firewalls for connection failover in case of reboots (but not with SMB). I just want to say, there are solutions out there! @shEiD Use the right tool for the right task. My opinion! Learn Linux administration, all the tools I mentioned are free of charge, Unraid costs money, CEPH not; but professional support can be bought. Proxmox has a GUI for CEPH management. CEPH also has an own Dashboard if you want to use CEPH standalone, but for advanced management, you need to learn Linux administration. For performance, see this: https://www.proxmox.com/en/downloads/item/proxmox-ve-ceph-benchmark PS: yes the thread is old, but for someone returning and being unhappy with Unraids limits (for example like @Lev), I would like to have some solutions written here to be sure nobody is unhappy with Unraid not knowing there is another tool for the task. Unraid is great, but I cannot repeat it enough: Use the right tool for the right task. If you have a hammer, everything looks like a nail. But screws need a screwdriver.

Thank you very much, that is very nice from you!

I am not sure if this was asked already, but I would like to suggest to be able to disable logging and history completely. The idea is that when you have a fully encrypted array with your business and private documents (e.g. invoices, customer data like works, ...) then your metadata and files are safe on the encrypted array against theft or drive failure warranty processes, but when you use unbalance to spill or gather the mentioned data, the metadata of the files is written on the unprotected usb flash drive within the history and the log file, thus leaking business and personal data to an unencrypted area. It would be much safer to be able to disable the log and history or hold it completely in memory until reboot but not write it towards the flash drive. Currently, manual deletion is the only way to get rid of these traces.

Is there any progress in this case? I would be happy to know! Thank you! by the way (commenting the post by tr0910): For multiple disks combined, it would propably work if one can not only tune the smartctl command per disk for one drive but for multiple drives per disk (like adding multiple smartctl command-parameters per disk)... I guess that could be a seperate request for later...

Thank you a lot! That would be perfect as it allows everyone to finetune the smartctl commands for his/her specific needs!

Information about the "twaN" device can be found in the man-page of smartctl: http://manpages.debian.org/cgi-bin/man.cgi?query=smartctl I am quoting the relevant part: 3ware,N - [FreeBSD and Linux only] the device consists of one or more ATA disks connected to a 3ware RAID controller. The non- negative integer N (in the range from 0 to 127 inclusive) denotes which disk on the controller is monitored. Use syntax such as: smartctl -a -d 3ware,2 /dev/sda [Linux only] smartctl -a -d 3ware,0 /dev/twe0 smartctl -a -d 3ware,1 /dev/twa0 smartctl -a -d 3ware,1 /dev/twl0 [Linux only] smartctl -a -d 3ware,1 /dev/tws0 [FreeBSD only] The first two forms, which refer to devices /dev/sda-z and /dev/twe0-15, may be used with 3ware series 6000, 7000, and 8000 series controllers that use the 3x-xxxx driver. Note that the /dev/sda-z form is deprecated starting with the Linux 2.6 kernel series and may not be supported by the Linux kernel in the near future. The final form, which refers to devices /dev/twa0-15, must be used with 3ware 9000 series controllers, which use the 3w-9xxx driver. The devices /dev/twl0-15 [Linux] or /dev/tws0-15 [FreeBSD] must be used with the 3ware/LSI 9750 series controllers which use the 3w-sas driver. Note that if the special character device nodes /dev/tw[ls]?, /dev/twa? and /dev/twe? do not exist, or exist with the incor- rect major or minor numbers, smartctl will recreate them on the fly. Typically /dev/twa0 refers to the first 9000-series con- troller, /dev/twa1 refers to the second 9000 series controller, and so on. The /dev/twl0 devices refers to the first 9750 series controller, /dev/twl1 resfers to the second 9750 series controller, and so on. Likewise /dev/twe0 refers to the first 6/7/8000-series controller, /dev/twe1 refers to the second 6/7/8000 series controller, and so on. Note that for the 6/7/8000 controllers, any of the physical disks can be queried or examined using any of the 3ware's SCSI logical device /dev/sd? entries. Thus, if logical device /dev/sda is made up of two physical disks (3ware ports zero and one) and logical device /dev/sdb is made up of two other physi- cal disks (3ware ports two and three) then you can examine the SMART data on any of the four physical disks using either SCSI device /dev/sda or /dev/sdb. If you need to know which logical SCSI device a particular physical disk (3ware port) is associ- ated with, use the dmesg or SYSLOG output to show which SCSI ID corresponds to a particular 3ware unit, and then use the 3ware CLI or 3dm tool to determine which ports (physical disks) corre- spond to particular 3ware units. If the value of N corresponds to a port that does not exist on the 3ware controller, or to a port that does not physically have a disk attached to it, the behavior of smartctl depends upon the specific controller model, firmware, Linux kernel and platform. In some cases you will get a warning message that the device does not exist. In other cases you will be presented with 'void' data for a non-existent device. Note that if the /dev/sd? addressing form is used, then older 3w-xxxx drivers do not pass the "Enable Autosave" ('-S on') and "Enable Automatic Offline" ('-o on') commands to the disk, and produce these types of harmless syslog error messages instead: "3w-xxxx: tw_ioctl(): Passthru size (123392) too big". This can be fixed by upgrading to version 1.02.00.037 or later of the 3w- xxxx driver, or by applying a patch to older versions. Alterna- tively, use the character device /dev/twe0-15 interface. The selective self-test functions ('-t select,A-B') are only supported using the character device interface /dev/twl0-15, /dev/tws0-15, /dev/twa0-15 and /dev/twe0-15. The necessary WRITE LOG commands can not be passed through the SCSI interface. So basically, the /dev/twaN device does not exist in the system after boot and is created automatically by smartctl after the first call. The first call of the command "smartctl -d 3ware,13 -a /dev/twa0" after boot does not work and says that the /dev/twa0 device does not exist - but the smartctl tool creates the device and the second call of the same command (and any further calls) do work exactly like they should. The /dev/twaN device is for the 3ware 9000 Series - other series do have other devicenames, so my suggestion for having the possibility to overwrite the device per disk (to /dev/twa0 for example) and to have the ability to add additional parameters (like -d 3ware,13) per disk would solve the problem in my case and also be flexible enough for other users to overwrite with the device they need (like /dev/twe0 for 6/7/8000 series) and with the parameters they need (like -d areca,N for another manufacturer). It would be important to have that flexibility, for example when you have multiple RAID controllers and need to address via /dev/twa0 and /dev/twa1 for different disks at different controllers. So the /dev/sdN device is the correct device for mounting, writing and doing everything concering the filesystem. But the /dev/twaN device (for refering the RAID controller) is needed for SMART only as these devices are created via smartctl. So I can use the unRAID correctly with the /dev/sdN devices, but cannot see the SMART values but only the error I already posted. "you may need to replace /dev/sdb with /dev/twlN, /dev/twaN or /dev/tweN". (Well I can call the values via SSH access manually, but who does want to call all SMART values from every disk via SSH when you have a Web-Interface) Hope that helps!

Hi, I would like to request a feature concering the unRAID 6 webUI. I would like to change the smartctl call to another one for getting it to work with a hardware RAID controller. Several years ago, there was unMENU with this thread: http://lime-technology.com/forum/index.php?topic=12228.0 In unMENU, one could set a "smartopt" variable. I never had unRAID 5 with unMENU, but that is the information I found. Now in unRAID 6, the smartctl commands are fixed inside the webUI - if you do a search in the github repository, you can see that: https://github.com/bergware/dynamix/search?q=smartctl The error I am getting is the following: "you may need to replace /dev/sdb with /dev/twlN, /dev/twaN or /dev/tweN" So the smartctl call to /dev/sdb says: root@Tower:~# smartctl -a /dev/sdb smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.0.4-unRAID] (local build) Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org Smartctl open device: /dev/sdb failed: AMCC/3ware controller, please try adding '-d 3ware,N', you may need to replace /dev/sdb with /dev/twlN, /dev/twaN or /dev/tweN But when I change the call like it is proposed, I see the correct output: root@Tower:~# smartctl -d 3ware,13 -a /dev/twa0 ... correct output ... I need to change the call to /dev/twaN for all /dev/sdX calls AND I need to set the additionals parameter "-d 3ware,N" for each device where N is the device number in the hardware controller. So I need an input field where I can set an additional "-d controller,N" parameter for all smartctl calls the webUI does and I need a field for overwriting the /dev/sdX call with another device. I am sure that would be an easy change inside the webUI, as two additional fields for configuring each disk and a change to the few smartctl calls I found via the github search honoring the additionals fields are no big deal. I would be glad if that could be implemented so that I can see all smartctl values in the webUI. Thanks a lot!