gfjardim

This is obviously a bug, but it's harmless. It's symptom of some leftover files on your system. I'll investigate as soon as possible, ok? Thanks for reporting. In Unassigned Devices, you can see a X icon next to the disk serial which can be used to clear the old session. In Tools > Preclear Disk, that icon exist too with same functionality. FYI for Mac users - I was experiencing this issue as well. I have a slot dedicated to pre clearing disks for this and other NAS I have. In Safari, clicking the red X on the Preclear plugin page did nothing. Switched to Chrome and it acted just as you said it should. Not sure why it behaves different in Safari. Side question: Can the plugin be modified to read the serial and if it doesn't match the last preclear on record, it behaves as if no prelcear has been done and presents users with "Start Preclear"? Thanks for this awesome tool! I'm macOS user too, and I can't replicate the issue. Every time a user complains about it, I open Safari and it works strait forward. Can't patch what I can't replicate, I'm afraid. If you can invite me on a remote TeamViewer session, than I can investigate it using your setup. I first mentioned this twelve pages ago. Whoops, sorry about that . I did read back several pages in the thread, but admittedly could have searched for "Safari". Thx for pointing out this has already been identified. I'm pleased that someone else mentioned it, actually. Browser issues often get overlooked, especially when they affect only a small minority of users. I'm sorry but I couldn't track down the issue yet. I get this using version 2017.01.28 on two installs. This is a common error, apparently. I'll make some changes in the code ASAP. Thanks for reporting.

Was this feature removed? I'm using latest preclear and trying to stop the array during a preclear doesn't work anymore, stuck on sync filesystems... PS: Any way to check preclear progress using the console? Just run "preclear" on the console. Fixed!

In Unassigned Devices, you can see a X icon next to the disk serial which can be used to clear the old session. In Tools > Preclear Disk, that icon exist too with same functionality.

but between work, Christmas preparations, and kids visiting, I haven't had time to research it. When I finally did, the answer is already here! The real prompt for figuring out what the heck was going on, though, was the prompt to install the Statistics Plugin with a mention of TOR. Having no recollection of ever having unRAID prompt me to install anything before (as opposed to the auto-update, which is awesome), I was quite concerned. The good news is that a search for "statistics" in the plugin support forum brought me directly to the answer. I'd suggest that you may want to modify the pop-up for future users to tell them that this is coming from the pre-clear plugin and add a link to the pop-up itself that takes users directly to the support page. Done. Thanks for your suggestion.

In the next update a Erase option will be available. It will use random generated data to fill the disk, making drive wipe more secure. I'm pretty sure you will make a good use of it.

Just updated the plugin. You will be asked to install "Statistics" after update.

You brought to my attention this problem with corporate rules, and I think it's very positive for the plugin development. I'll remove the report functionality from the plugin altogether, and make a separate plugin for that. Upon installation, users will be prompted if they want to install the new plugin or not. This will avoid the download of TOR inside corporate facilities and avoid this kind of situation. My deepest apologies for the mess it created.

The firewall is only open at certain days/times and only when certain conditions are met. The firewall controls are separate from the rest of the network. The firewall slams shut if certain kinds of traffic are detected (TOR being one of those) Unraid is not work's primary backup. It is an onsite backup to the backups. We replaced our Ubuntu based backup years ago as Unraid was more stable for this function. We are required to have this onsite backup and it must use a different OS/filesystem from the primary data. CA Auto Update is scheduled to run at a time when other machines have access to reach out for updates while the firewall is open. As such, no user has access to touch files. A script from another machine runs via SSH which stops samba, unmounts each disk, etc The firewall will not open if samba is detected on the network. Before the start of the day, samba and the array is restarted ... I do not have access to manage the firewall or the Samba policies/protocols -> that belongs to a separate independent entity. … Back to the original question. Can the preclear data collection be Opt-In… or at least an option to Opt-Out? It is Opt-In. The Tor service is only activated while uploading the stats. What do you need to satisfy your company's policy?

Do you mean that you physically move the drive without shutting down the server? I stop the array, pull the drive, relocate the drive, restart the array. So when I hover over the red X, the tooltip says "Stop Preclear" (obviously there isn't one ongoing) and clicking it refreshes the page, but doesn't actually change anything that I can see - I've still no option to start a preclear. Pretty sure by now that Im missing something obvious, but not sure what it is! Is it mounted by Unassigned Devices?

This is because you removed the old drive and the Linux UDEV subsystem reused it's device name. You can easily remove the old instance by clicking the X icon next to the new drive name.

This is tor f#ckin up the logrotate directory. I'll change the build script ant update the package. Thanks for reporting.

Guys, the statistics report system has been put in place. The great think about scripts is that they allow anyone to see what happens in the background. I'll comment some technical details to make this as transparent as possible. When finishes or fails in some of the stages the preclear script will write a ".stats" file under /boot/config/plugins/preclear.disk/ directory. This file is a multi-level INI file and each of it's sections contains a information that can get exported to Google Forms. Each section has an entry variable which corresponds to the desired field in "Google Forms". It also has a title variable which corresponds the webui title and a value variable which contains the actual data. The relevant function in script can be found here. When Unassigned Devices or Preclear Disk plugins gets loaded by the web interface, the getOpenStatistics() javascript function will probe for any existing ".stats" files on the plugin config directory (here). If a report is found, it will trigger an alert asking the user to Send or Cancel that report. If you hit Send, the getOpenStatistics() function will send a POST request with "action=send_statistics" form-data to "Preclear.php". That request will be processed by this portion of the code; the script will start the TOR service, compare both local external IP and the TOR remote IP (and exit if they match), create the curl request command and run it, stop the TOR service and remove the ".stats" file at the end. If you hit Cancel, the getOpenStatistics() function will send a POST request with "action= remove_statistics" form-data to "Preclear.php", and the script will remove the ".stats" file. I hope this will remove any doubts about this functionality, and I do expect for your statistics!

Tried with ie edge, ie 11, firefox and chrome. None works. Any ideas? Please update the script and try it again.

Could you change the google form type to log the IP address, so that we can see that what is logged is not our own IP? On that same note, are you planning to allow us to peruse the data that you collect on the google form? If you say no, I understand, and don't necessarily object, but it would be nice if you would add a link to the form data on the submission confirmation after you send data. I don't think it's necessary, the plugin does the comparison by itself: case 'send_statistics': $file = $_POST["file"]; $stat = parse_ini_file($file,true) ?: []; shell_exec("/etc/rc.d/rc.tor start 2>&1"); $myip = trim(shell_exec("curl -s http://whatismyip.akamai.com 2>/dev/null")); $torip = trim(shell_exec("curl -s --socks5-hostname 127.0.0.1:9050 http://whatismyip.akamai.com 2>/dev/null")); if ($myip == $torip) { exit(); } shell_exec("echo sending preclear report:|logger -t'preclear_disk'"); shell_exec("echo my ip = $myip|logger -t'preclear_disk'"); shell_exec("echo tor ip = $torip|logger -t'preclear_disk'"); $cmd = "curl -s --socks5-hostname 127.0.0.1:9050 \"https://docs.google.com/forms/d/e/<id>/formResponse\" -d ifq"; foreach ($stat as $key => $value) { $cmd .= " -d \"{$value['entry']}=".str_replace("^n", "\n", $value['value'])."\""; } shell_exec("$cmd 2>&1"); shell_exec("/etc/rc.d/rc.tor stop 2>&1"); @unlink($file); break; I'll put the form URL and the respective spreadsheet in the main OP, so anyone can use it.

Any way to anonymize the IP address? Not that I'm aware of if you continue to use google. One reason google is "free" is because they collect every bit of data they can on every user and computer they can. Cross referencing all the little tidbits allows them to sell user profiles as a "product" to advertisers. They can't do that if they can't correlate info somehow. As long as you disclose that google is the one receiving and holding the form data, I see no issue with it. The data really is anonymous to you, and anyone else accessing the forms from outside. It's only google's internal workings that do the correlation and data mining. Maybe TOR? I'm able to make use of TOR to anonymize the IP address. Is that ok for you, guys?

Any way to anonymize the IP address? Not that I'm aware of if you continue to use google. One reason google is "free" is because they collect every bit of data they can on every user and computer they can. Cross referencing all the little tidbits allows them to sell user profiles as a "product" to advertisers. They can't do that if they can't correlate info somehow. As long as you disclose that google is the one receiving and holding the form data, I see no issue with it. The data really is anonymous to you, and anyone else accessing the forms from outside. It's only google's internal workings that do the correlation and data mining. Maybe TOR?

I'd like to see google mentioned, since they are the entity that is actually receiving the data on your behalf. That way if I'm watching the bits flow through my router, I'll know with reasonable certainty that a connection from unraid to google is a result of the dialog I just clicked. The "anonymous" wording may be the end result to YOU, but it certainly isn't anonymous to google. Any way to anonymize the IP address?

I was thinking about presenting it on first open of Preclear/Unassigned Devices, and let users change it on the Help Section of Preclear Plugin page. Why do you think it should be done after each preclear session? Why not? That way if something did not seems right or there was a problem (or even a recently introduced size or disk model), the user could decide that in this case, he might want to upload the data. If you are going to provide an check-box type option, you might provide an "Ask me each Time" as a third choice to hard 'Yes' or 'No' choice. You are providing this software free to the user base and you do (in my opinion) have the right to be insistent in requesting an upload of the results. Absolutely. I accept as a user of your software that you ask certain things in return to help you. I just ask that you be open and transparent when you make those requests. I would be perfectly fine with a mandatory nag at the end of each preclear that asks "Will you help me improve this software by providing me with the statistical results and / or errors of this preclear?" What I will not stand for is the silent transmission of my information with no explicit permission. Will this work? How did I implemented it? 1) The script will write a ".stats" file on /boot/config/plugins/preclear.disk directory after it's done; 2) The webui will present that popup for each ".stats" file it found on /boot/config/plugins/preclear.disk directory; 3) ".stats" file will be deleted after sent, of after "Cancel" is clicked.

I was thinking about presenting it on first open of Preclear/Unassigned Devices, and let users change it on the Help Section of Preclear Plugin page. Why do you think it should be done after each preclear session?

As far as I am concerned, #2 is the only one that I would consider is not an invasion of my privacy. While I realize that the other information is nice to have, it is not essential to working of this product. While LimeTech 'knows' how many time its software is download (which I assume you also know) and how many licenses it has issued, it has no real knowledge of how its product is being used, actual numbers of disks and with what plugins, Dockers, or VM's. (There are a fair number of us with Pro licenses that aren't using anywhere near the numbers of disks to require it!) I realize that I have exposed some of this data in my 'signature' line of this post. However, the collection of that data would be very difficult and expensive process and even more difficult to tie directly to me as an individual. (The upload of the data you propose to Google will have my IP address as a part of that data which would serve Google's purposes for THEIR data collection! Read their terms of use...) A custom made solution for data collection doesn't worth the effort, so I guess IP recording is inevitable (but every site/service track us nowadays even this forum). I agree you have all the right to choose if your data is uploaded or not, so I'm making this optional. Is something like this enough?

What I expect to collect: 1) number of disks that are precleared (dying to know ); 2) any silent bugs that gets reported on the logs; 3) average size of disks that gets precleared, their model and the average speed and time we should expect to that model; 4) success rate; 5) rate of disks with SMART problems;

Where is this data going? If it is to be transmitted back to you, will there be an opt-out setting? To Google Forms. Since all data is anonymous (no serial exposed, for example), is there a real need for an opt-out setting? To be brutally honest, I am very suspicious of any type of application that phones-home. Almost everyone of them promises that all Data is be anonymized but it has often been found later that was not the complete case. I would prefer to have an opt-out option but at least in this case, I have the option to doing the preclear by using the old-old shell script. (I admit that I am paranoid and that I have disable both Flash and Java on all of my household computers. I refuse to link e-mail servers together and/or any of the social media sites together. I avoid clicking on 98% of the links that I am sent by 'Friends' until I have verified them from a second source. But then I have never been hit by a virus or malware to date...) Since you fell strong about it can add a toggle to disable the report. These are the statistics I expect to collect: https://docs.google.com/spreadsheets/d/1zERJQlFWGgMGKnKgGUNKnguGUdCQ2g0XT9Q5yMJio6o/edit?usp=sharing

Where is this data going? If it is to be transmitted back to you, will there be an opt-out setting? To Google Forms. Since all data is anonymous (no serial exposed, for example), is there a real need for an opt-out setting?

Guys, I'm considering add some sort of statistics report inside the plugin script. This will allow me to track potential bugs and performance data. Initially I'm thinking in gathering model, size, smart device type, total elapsed time, log and if it was successful or not. Any concerns and/or suggestions?

Awesome video, gridrunner, added to OP if you don't mind.