unRAID OS version 6.4.0 Stable Release Available

[CowboyRedBeard]

Hey, I don't mean any offense guys. Just trying to offer my experience.

 

Overall the webGUI is 1,000x better... way more responsive on my old eBay junker I'm running this on and that's a huge positive by itself. That was always a source of frustration.

[limetech]

10 minutes ago, CowboyRedBeard said:

Hey, I don't mean any offense guys. Just trying to offer my experience.

No offense taken, however ...

 

51 minutes ago, CowboyRedBeard said:

That's a really poor idea for a ton of reasons.

It's very possible we have overlooked something.  We would be very interested in everything you find objectionable.

 

44 minutes ago, CowboyRedBeard said:

being reliant on their DNS for it to work. That's enough of a reason by itself.

Sure but it's hosted on redundant Amazon EC2 servers, not some in someone's basement.

If the servers go down or your internet goes down and all your local caches are flushed you can make an entry for the FQDN in your clients hosts file as last resort.

[limetech]

19 hours ago, jonathanm said:

As soon as we can get limetech to add a long smart test to the end of a new disk inclusion cycle after the zeroing is done, I think it can be relegated to those who want to work through the command line version manually. No need to keep trying to integrate it into the GUI as a graphical plugin.

Why not do the long test before assigning into array?

[pwm]

1 hour ago, CowboyRedBeard said:

A self signed cert you can add as trusted would be better IMO.

That can be debated, because that requires that the user knows how to do this. And that the user performs that action on phone, workstation, laptop, ... that is expected to reach the web interface.

 

A semi-good solution that is simple enough that it works for the majority of people is often better than "the best" solution that lots of users will not use.

[pwm]

1 hour ago, CowboyRedBeard said:

The only other humble advice I'd offer for the future is... next time maybe have a "are you sure you want to upgrade?" verification.

Yes - operations that updates the system really should ask a "Are you sure you want to update?" to avoid accidental updates.

[CowboyRedBeard]

34 minutes ago, limetech said:

No offense taken, however ...

 

It's very possible we have overlooked something.  We would be very interested in everything you find objectionable.

 

Sure but it's hosted on redundant Amazon EC2 servers, not some in someone's basement.

If the servers go down or your internet goes down and all your local caches are flushed you can make an entry for the FQDN in your clients hosts file as last resort.

 

For me, and I'm sure a lot of other users, I'm primarily using unRAID as a way to acquire / store / serve up media that I download on the interwebz. I'd be concerned that there is someplace (via logs or something else) a centralized repository available for certain litigious organizations to go hunting with. They're certainly always looking for new and creative ways to find people to sue.

 

Not sure what all info you guys store in that hosting, but anything like that also makes a pretty attractive target for malicious exploration by nefarious groups.

 

The operational aspect of it being externally hosted is unnecessary, and introduces delay and potential for failure IMO. But I understand that was a balance of convenience for users.

 

Perhaps a slightly more sophisticated integration with a partner organization allowing the user to create their own, configurable cert. For example, I often jump into my unraid box while at my office and I've been running it through VPN because the previous lack of SSL. So having a publicly accessible SSL site would be cool, as an option but I understand that's not everyone's use case.

 

I mean, most folks that are using this are probably capable of such a thing... at least if there is some modest level of guidance. I'd guess 90%+ of unraid users are modestly technical at least. Me for example, I'm not an advanced *nix user but I can read directions and have a enough general IT experience I could figure it out.

 

That's just a few thoughts.

 

But like you guys said, it's not mandatory and self or externally signed certs are an option.

 

 

 

Edited January 15, 2018 by CowboyRedBeard

[gar13]

21 hours ago, Squid said:

Yeah, IDK.  NFS sharing just up and wanted more memory, it wasn't available, so it killed off something.  Weird thing though is that "notify" segfaulted a minute before that happened.  Don't know if they are related or not, but maybe run a memtest just to rule out bad memory.

Changed by NFS mount options on the client FSTAB from:

 

nfs rsize=8192,wsize=8192,timeo=14,intr

 

to:

 

nfs auto,nofail,noatime,nolock,intr,tcp,actimeo=1800 0 0

 

seems to have resolved the issue. The memory & CPU usage went way down and have been stable for about 14 hours. Thanks for the help yesterday.

[JonathanM]

1 hour ago, limetech said:

Why not do the long test before assigning into array?

Because not all sectors may have been recently written. After it's been cleared with all zeros you know each bit has been touched for writing, so a long smart test will verify that each of those freshly written bits can be read without error. Theoretically if the write completes with no error you are good, but it's nice to have a verification after the write, and logging a long smart test as successful at the beginning of a drive's tenure in the array is nice. Having that test be an optional step integrated into the process will ensure more people are taking an active step in verifying the drive is healthy enough to count on to recover another failed drive. It's also a few more working hours into the bathtub curve of infant mortality.

 

I think what you are getting at is "why should limetech invest time in programming changes that the user can do themselves manually", and I think the answer is, because too many people WON'T do it unless prompted, and something you can do with relatively little time investment towards protecting users from themselves is worth doing.

[limetech]

4 minutes ago, jonathanm said:

I think what you are getting at is "why should limetech invest time in programming changes that the user can do themselves manually",

 

LOL no, I was thinking that upon completion of clearing operation, which takes already takes a long time, user now has to wait an equivalent amount of time for the device itself to read the entire media.  During this time, what?  The device is considered part of the array, if the long test fails then there needs to be another control to remove the device from the array.

 

The assumption is this: the drive has already been written at the factory, presumably with zeros, but maybe not.  Probably the drive has also been read at the factory.  Personally I don't see any point in a "pre-clear" operation other than to prep a device that will go on the shelf, ready to install instantly to expand storage, but realize others have different, valid opinions.

[Cysix]

Nasty bug... upgrading to 6.4.0 reset the GUI port to 80.  Of course nothing indicated such, so I ended up having to DRIVE over to physically access the server.  Never again will I upgrade while physically remote from the server.

[JonathanM]

Just now, limetech said:

The assumption is this: the drive has already been written at the factory, presumably with zeros, but maybe not.  Probably the drive has also been read at the factory.

A large majority of issues with hard drives are with crappy handling after they leave the factory. It's scary how fragile a bare drive can be to certain specific impacts. Those of us who have been dealing with computers for a long time instinctively treat bare drives almost with reverence, but UPS, Fedex and USPS seem to take pride in how much they can abuse a package without showing visible damage.

 

7 minutes ago, limetech said:

I was thinking that upon completion of clearing operation, which takes already takes a long time, user now has to wait an equivalent amount of time for the device itself to read the entire media.

They don't HAVE to wait. An optional confidence test logged on the drive's own smart system is worth the hours invested to many users, to those who would rather save the time, they can always toggle the smart test to OFF. The drive can still be available during the smart test, but the cautious user will wait for a good test result to start populating it.

 

7 minutes ago, limetech said:

The device is considered part of the array, if the long test fails then there needs to be another control to remove the device from the array.

As you implied, probably better than 99% of the drives are going to pass the smart test if they don't kick an error during the clear, if the long test fails or is aborted notify the user, but let the normal red ball logic handle drive inclusion. If the user ignores the failure notice and fills the drive with data, it's probably going to red ball anyway when it hits the bad sector.

 

In my opinion from reading the forums over the years, Joe L.'s preclear script and successors was used primarily as a drive confidence test, secondarily as intended for quick array insertion. It became almost a gold standard in drive testing, at least around here. If the drive passed 3 full preclear cycles, you could almost guarantee 100% a long and productive life for it. A red balled drive would be subjected to a preclear before throwing in the towel and sending it in for replacement or scrapping it.

 

Having the option for unraid to automatically do a confidence test before you start trusting the drive with all your data is a worthy addition to me. People tend to forget that a failing drive, even an empty one, puts the whole array at theoretical risk, since any additional failure beyond parity tolerance means losing all failed drives.

[itimpi]

9 minutes ago, Cysix said:

Nasty bug... upgrading to 6.4.0 reset the GUI port to 80.  Of course nothing indicated such, so I ended up having to DRIVE over to physically access the server.  Never again will I upgrade while physically remote from the server.

It is not a bug as it is mentioned in the Release notes as one of the changes that you no longer set the port in the go file (but maybe not prominently enough?).

Edited January 15, 2018 by itimpi

[limetech]

8 minutes ago, Cysix said:

Nasty bug... upgrading to 6.4.0 reset the GUI port to 80.  Of course nothing indicated such, so I ended up having to DRIVE over to physically access the server.  Never again will I upgrade while physically remote from the server.

 

It's in the OP Release Notes under Secure Access.

[JonathanM]

9 minutes ago, Cysix said:

Nasty bug... upgrading to 6.4.0 reset the GUI port to 80.  Of course nothing indicated such, so I ended up having to DRIVE over to physically access the server.  Never again will I upgrade while physically remote from the server.

You could have logged in SSH and fixed it that way.

[limetech]

9 minutes ago, jonathanm said:

It became almost a gold standard in drive testing, at least around here. If the drive passed 3 full preclear cycles, you could almost guarantee 100% a long and productive life for it.

 

Agreed, but I thought you were advocating for adding the SMART long test at end of the clear operation that takes place if you add a non-pre-cleared device to an existing array.  That would mean only a single read pass, which is a sequential read pass without any seeks thrown in there just to rattle the drive.  If this is all you wanted, why not simply start a parity-check after installing a new device in this manner?

 

Seems to me if you want to exercise new disks you still want an offline utility for doing so that leaves it in "pre-cleared" state.

[Dr_Cox1911]

On 12.1.2018 at 9:23 PM, limetech said:

You will also notice in your address bar a very funny looking URL consisting of a 40-hex-character subdomain of unraid.net. We have set up a LimeTech DNS server that will resolve that URL to your servers IP address on your local network. That FQDN is unique to your certificate. When your browser resolves that URL it is given your local IP address which it then uses to perform the https connection handshake.

Is this on per default? So right after I update my server IP will be added to the LimeTech DNS?

If so, any way to alter this to the local, self-signed cert right away and don't add it to the DNS?

[Tybio]

Upgraded, no issues.  Thanks for the great release!

[ufopinball]

Painless upgrade, uptime is 12+ hours on my Ryzen 1800X.  I've not dipped into the newer 6.4.0 features, but am enjoying the NTP (Nested Page Tables) fix for better GPU performance in my Windows 10 VM.  Thanks for all the efforts!!

[pwm]

1 hour ago, jonathanm said:

Theoretically if the write completes with no error you are good

Actually not, since the drive does not perform any action to verify that the write completes with no error. The drive is happy if it manages to locate the identification for the correct sector and doesn't detect a g-sensor bump or high-fly while doing the write. The drive just assumes that the surface area is good enough that there will not be more errors than what the ECC can recover while still have a bit of safety margin for ageing of the data.

 

So while it statistically means you are good if the write completes with no error, there are multiple ways the write can actually have failed.

 

 

1 hour ago, limetech said:

the drive has already been written at the factory, presumably with zeros, but maybe not.  Probably the drive has also been read at the factory. 

The factory most definitely have performed both writes and following reads. But the tests at the factory will not tell if the drive has somehow managed to touch down the head on the surface after the factory tests and before the user starts to store critical data on the disk. Drives may specify that they can handle 100-200g when not spinning but it is quite easy to produce impulses stronger than that when two hard objects hit each other.

[dlandon]

2 minutes ago, limetech said:

 

Agreed, but I thought you were advocating for adding the SMART long test at end of the clear operation that takes place if you add a non-pre-cleared device to an existing array.  That would mean only a single read pass, which is a sequential read pass without any seeks thrown in there just to rattle the drive.  If this is all you wanted, why not simply start a parity-check after installing a new device in this manner?

 

Seems to me if you want to exercise new disks you still want an offline utility for doing so that leaves it in "pre-cleared" state.

The operative word is "offline".  Trying to include a preclear operation in the webgui with UD working with the disks not in the array is causing me a lot of support issues and gfjardim a lot of issues with keeping the preclear from hanging.

 

Preclearing a disk is not a daily need for unRAID operation and it should be done on a backup or dedicated system using the script as in the Joe L way of doing things.  Those of you that preclear a gazillion disks can do it on a dedicated system and not on unRAID.  New users think preclear is a necessity and install the plugin only to find out the issues with the preclear operation on unRAID and then ask for all kinds of support.  Just install the disks, let unRAID clear them, and run a parity check if you are nervous about the state of the disk.  A new disk should not need a preclear.  How many people run a preclear on a new disk in a Windows PC?

[JonathanM]

33 minutes ago, limetech said:

If this is all you wanted, why not simply start a parity-check after installing a new device in this manner?

Because a long smart test logs itself to the drive's firmware, and does more internal testing than just a full read would do.

 

As a matter of practice I personally do a non-correcting check after messing around with the server guts, it's a relatively quick confidence check that I didn't mess anything else up while I was in there. A long smart test is a different tool with similar results, and the status stays logged internal to the drive for future reference regardless of the platform.

 

33 minutes ago, limetech said:

Seems to me if you want to exercise new disks you still want an offline utility for doing so that leaves it in "pre-cleared" state.

For the truly hard core, there are options still available for doing just that. I'm advocating for slightly more strenuous drive testing automatically offered to everyone out of the box by default without messing with poorly integrated GUI plugins or command line work.

 

In an ideal world, Limetech would integrate and support the preclear utility in a fully fleshed out GUI tool option. Years and years have passed, and it still hasn't happened, so I'm asking for baby steps to help steer the less technical to test their drives before trusting them.

Edited January 15, 2018 by jonathanm
poor wording

[pwm]

4 minutes ago, dlandon said:

How many people run a preclear on a new disk in a Windows PC?

I have had multiple Windows disks fail.

But I try to keep down the storage size for the system disk so it's easy to backup. And then separate installation from data storage. So the Windows machines mostly stores dynamic information that is easy to restore.

 

The special thing with a larger file server is that it handles so large amounts of data that it is a huge task to perform a major restore. And that restore affects the redundancy from parity so a single disk affects the reliability of the storage on the other disks too. I can restore a Windows installation in a very short time. Restoring 50-100 TB because I got in a situation where parity isn't enough would be a quite significant task - especially since I after restore have to verify the hashes for all recovered files.

 

The same moment you start to think RAID, parity etc, you really have to analyze all parts of the chain to spot any weak links.

[JonathanM]

12 minutes ago, dlandon said:

A new disk should not need a preclear.

The vast majority don't need one, they are good to go.

I want to identify the one drive that hit the concrete from 3 meters in the air when the UPS sorter decided to spit the box off the track before I trust it with protecting the rest of my drives.

[Frank1940]

2 hours ago, jonathanm said:

The vast majority don't need one, they are good to go.

I want to identify the one drive that hit the concrete from 3 meters in the air when the UPS sorter decided to spit the box off the track before I trust it with protecting the rest of my drives.

+1

and protect against the ~1% that will fail in the first forty or so hours.  

 

I can also remember that a VERY large supplier of computer parts that would put a drive remove from a carton of an OEM-pack  (In basically a static-protection envelope) and toss it in a another box without so much a an scrap of bubble wrap and ship it to me!  I only got a couple that way and they both worked but I stopped buying there. 

 

EDIT: Plus, I usually have a spare drive that I buy as soon as I put a replacement drive in (or add a new new drive).  I want to thoroughly test that drive while I am in the vendor's return window as they have always provided a prepaid shipping sticker and a quick turn around. 

Edited January 15, 2018 by Frank1940

[JuanH]

Upgraded to 6.4 as well. Everything seems to load just fine (Disks, shares, Docker applications) except for the unRAID GUI. Can someone help me troubleshoot? tower-diagnostics-20180115-2234.zip.

Edited January 15, 2018 by JuanH