unRAID Server Release 6.2.0-beta20 Available


Recommended Posts

Starting VMs independent of the array is a feature that has been requested allot and one I'd love to facilitate running pfsense. Did this "feature" make it into 6.2?

This is not going to happen anytime soon, if ever.  It interferes with features we have planned for the future.  "Array Start/Stop" is really a misnomer.  In this context "array" refers to the entire set of devices attached to the server, not just the ones that are assigned to the parity-protected devices.

So do you have a solution in mind for those who wish to host a pfsense or other end point firewall appliance as a VM on unraid?
Link to comment

Starting VMs independent of the array is a feature that has been requested allot and one I'd love to facilitate running pfsense. Did this "feature" make it into 6.2?

This is not going to happen anytime soon, if ever.  It interferes with features we have planned for the future.  "Array Start/Stop" is really a misnomer.  In this context "array" refers to the entire set of devices attached to the server, not just the ones that are assigned to the parity-protected devices.

So do you have a solution in mind for those who wish to host a pfsense or other end point firewall appliance as a VM on unraid?

Sorry, but I dont see the problem.

Link to comment

Starting VMs independent of the array is a feature that has been requested allot and one I'd love to facilitate running pfsense. Did this "feature" make it into 6.2?

This is not going to happen anytime soon, if ever.  It interferes with features we have planned for the future.  "Array Start/Stop" is really a misnomer.  In this context "array" refers to the entire set of devices attached to the server, not just the ones that are assigned to the parity-protected devices.

So do you have a solution in mind for those who wish to host a pfsense or other end point firewall appliance as a VM on unraid?

 

It would seem that the "solution" is to use a small dedicated box outside of UnRAID if you need that functionality running when UnRAID isn't running.    If you're using UnRAID as an "all in one" box with no other PC's on the network, it would seem that it's not a problem if the pfsense box isn't running when UnRAID is shut down;  but if there are other devices (as is likely almost always the case) then an UnRAID VM doesn't seem to be the best approach.

 

 

Link to comment

Starting VMs independent of the array is a feature that has been requested allot and one I'd love to facilitate running pfsense. Did this "feature" make it into 6.2?

This is not going to happen anytime soon, if ever.  It interferes with features we have planned for the future.  "Array Start/Stop" is really a misnomer.  In this context "array" refers to the entire set of devices attached to the server, not just the ones that are assigned to the parity-protected devices.

So do you have a solution in mind for those who wish to host a pfsense or other end point firewall appliance as a VM on unraid?

 

What is the issue exactly?

Link to comment

Starting VMs independent of the array is a feature that has been requested allot and one I'd love to facilitate running pfsense. Did this "feature" make it into 6.2?

This is not going to happen anytime soon, if ever.  It interferes with features we have planned for the future.  "Array Start/Stop" is really a misnomer.  In this context "array" refers to the entire set of devices attached to the server, not just the ones that are assigned to the parity-protected devices.

 

That is such a shame. I believe you are making a mistake. If anything, ensuring that a user does NOT have to have a working array to use features of unRAID may bring MORE users to use your OS.

 

Starting VMs independent of the array is a feature that has been requested allot and one I'd love to facilitate running pfsense. Did this "feature" make it into 6.2?

This is not going to happen anytime soon, if ever.  It interferes with features we have planned for the future.  "Array Start/Stop" is really a misnomer.  In this context "array" refers to the entire set of devices attached to the server, not just the ones that are assigned to the parity-protected devices.

So do you have a solution in mind for those who wish to host a pfsense or other end point firewall appliance as a VM on unraid?

Sorry, but I dont see the problem.

 

One can install pfsense and run it within an unRAID hosted VM and when the Array is functional and working everything is fine. The problem comes when one has to take the Array down for maintenance / updates OR it goes down without user interaction (e.g. crashes etc).

 

As pfsense is the router for the network, without the VM running you will essentially loose your router and your connection to the Internet as well as the rest of the network and your unRAID box itself.

 

Starting VMs independent of the array is a feature that has been requested allot and one I'd love to facilitate running pfsense. Did this "feature" make it into 6.2?

This is not going to happen anytime soon, if ever.  It interferes with features we have planned for the future.  "Array Start/Stop" is really a misnomer.  In this context "array" refers to the entire set of devices attached to the server, not just the ones that are assigned to the parity-protected devices.

So do you have a solution in mind for those who wish to host a pfsense or other end point firewall appliance as a VM on unraid?

 

It would seem that the "solution" is to use a small dedicated box outside of UnRAID if you need that functionality running when UnRAID isn't running.    If you're using UnRAID as an "all in one" box with no other PC's on the network, it would seem that it's not a problem if the pfsense box isn't running when UnRAID is shut down;  but if there are other devices (as is likely almost always the case) then an UnRAID VM doesn't seem to be the best approach.

 

While this is a solution, it is not the one that I was looking for.

 

I don't want another piece of hardware on my network I want to utilise what I have in spare capacity with the hardware I have already purchased (which is consistent with unRAID's goals btw).

 

I think practically (if the ability to run a VM on system start is never enabled) the only solution is going to be have the Array & pfsense VM on Autostart. To deal with times when the Array needs to go down one can have a small low powered router with basic settings (IP allocation table, Internet connection) mirrored that can be used (as in WAN cable out of pfsense port and into router) to access unRAID and Inter short term.

 

Alternativley one can just set non DHCP static IP addresses on both the unRAID box (running pfsense) and a (1 or more) wired computers which will allow the two machines to communicate via a switch anyway and thus connect to and be able to maintain unRAID while the router is down.

Link to comment

Starting VMs independent of the array is a feature that has been requested allot and one I'd love to facilitate running pfsense. Did this "feature" make it into 6.2?

This is not going to happen anytime soon, if ever.  It interferes with features we have planned for the future.  "Array Start/Stop" is really a misnomer.  In this context "array" refers to the entire set of devices attached to the server, not just the ones that are assigned to the parity-protected devices.

So do you have a solution in mind for those who wish to host a pfsense or other end point firewall appliance as a VM on unraid?

 

What is the issue exactly?

 

Please see my above post.

Link to comment

Since 6.2 I have a very annoying bug that leads to unresponsiveness of unRAID in a dimension that I have to do a reset of the computer. No normal shutdown is possible anymore, not at all.

 

unRAID OS Version:

6.2.0-beta20

 

Description:

Filesystem hangs completely. Local file operations also are not possible anymore. Nothing that writes or reads files on the disks does work anymore.

Shutdown impossible, WebGUI still responsive until one uses a command that writes or reads a disk, like clicking on a FileBrowser Icon.

Only hard reset possible. SSH login still works, but any shutdown commands that try to sync or unmount the filesystem immediately hang.

 

How to reproduce:

Safe mode used! No plugins loaded.

Copy a file from one disk to another (not same disk, different disks!). The problem ONLY occurs, when reading and writing appear on the same time. Parity Check works so far.

Best results with copying a file via network (like SMB) - it takes about 10 seconds until the throughput drops to zero and the filesystems hang. So at the beginning, the copying works, but after about 10 seconds it stops completely. What worked so far was copying via SSH from one disk to another - but when you access another file from a third disk (for example), after about 10 seconds the filesystem start to hang - so I tried to copy a big file between two disks via SSH and read another big file from a third disk - hangs. Or copy via network between two disks hangs.

 

Expected results:

No hanging at all.

 

Actual results:

Hangs until hard reset and reboot.

 

Other information:

Logfiles added. The problem did not exist in 6.1 as long as I remember.

Reading one file at a time or writing one file at a time works flawless.

 

UPDATE:

There seem to be some misunderstandings: The logfile was created while the system was hanging for 4 minutes already. So the error happened before the diagnostics were created. That was a requirement for creating an error post in this thread, to generate the diagnostics after the system error happened and before the system was rebooted.

Indeed there are NO signs of the error at all in the logfile. I even started samba-logging, but it just stops logging and no error is visible. In syslog there also is nothing to see, but the disks all are unresponsive.

I was asked for SMART values. One can see in the logfiles that I set the SMART configuration correctly for my controller card. I can see all SMART values in the WebUI, can do tests and see all raw values. When diagnostics do not export the values though I configured the SMART correctly for every disk, this is obviously a bug in the diagnostics software that does not honor the SMART configuration. Nevertheless, all disks are running and have a green thumb up.

 

More updates here: https://lime-technology.com/forum/index.php?topic=47875.msg460756#msg460756

tower-diagnostics-20160331-2347.zip

Link to comment

Starting VMs independent of the array is a feature that has been requested allot and one I'd love to facilitate running pfsense. Did this "feature" make it into 6.2?

This is not going to happen anytime soon, if ever.  It interferes with features we have planned for the future.  "Array Start/Stop" is really a misnomer.  In this context "array" refers to the entire set of devices attached to the server, not just the ones that are assigned to the parity-protected devices.

So do you have a solution in mind for those who wish to host a pfsense or other end point firewall appliance as a VM on unraid?

 

What is the issue exactly?

Well for me if I want to upgrade disks in the array from reiserfs and still run the VM (Windows in this case) I have to continually start and stop the VM as I start and stop the array to change the file system type.  Since my Windows VM is recording from TV tuners I have to wait for windows in the recording schedule before I can do that so it took a month and a half to convert to xfs.  My Windows VM has its own passed through controller for recording drives but the boot drive has to be an image (as far as I know) so I have to start and stop the VM with the array.  I can see other things like this in the future where it would be nicer to have an external drive for the VM that doesn't cause problems like putting the array in to maintenance mode to trouble shoot drive problems - do disk checks without having to shut down the VM.
Link to comment

The beta looks pretty good for most of my containers (emby is not playing video for me / tvheadend does play video)  my vm's (windows 10, lubuntu) don't start clean and I can't stop them either.  Most of the new features seem to function well.

If I read your config correct, ar least your Windows VM seems to be on a non-cached share? So its on the array?

If so it does sound like this: http://lime-technology.com/forum/index.php?topic=47875.msg459773#msg459773

 

It could be the reason your containers stop to work, because that thing pretty much locks up the array.

 

But I could be wrong, I am not that familiar with theese diagnostics files yet.

Link to comment

Your syslog is only 20 seconds long after a reboot, and there is no SMART for any disks. Try to get us something that actually includes the activities you are concerned about.

 

I updated my post with the following information:

 

UPDATE:

There seem to be some misunderstandings: The logfile was created while the system was hanging for 4 minutes already. So the error happened before the diagnostics were created. That was a requirement for creating an error post in this thread, to generate the diagnostics after the system error happened and before the system was rebooted.

Indeed there are NO signs of the error at all in the logfile. I even started samba-logging, but it just stops logging and no error is visible. In syslog there also is nothing to see, but the disks all are unresponsive.

I was asked for SMART values. One can see in the logfiles that I set the SMART configuration correctly for my controller card. I can see all SMART values in the WebUI, can do tests and see all raw values. When diagnostics do not export the values though I configured the SMART correctly for every disk, this is obviously a bug in the diagnostics software that does not honor the SMART configuration. Nevertheless, all disks are running and have a green thumb up.

Link to comment

As it's the first of the month and I have a scheduled parity check underway with "Write corrections to parity disk" set to the default "Yes" in the scheduler, I was wondering if it will do literally that - now that I have dual parity - or will it treat any error in a more intelligent way? With only one parity disk the assumption was that any inconsistency was due to a parity error, not a data error. But with P and Q parity there's scope to use the parity check information more wisely. Has that actually been implemented?

 

Link to comment

As it's the first of the month and I have a scheduled parity check underway with "Write corrections to parity disk" set to the default "Yes" in the scheduler, I was wondering if it will do literally that - now that I have dual parity - or will it treat any error in a more intelligent way? With only one parity disk the assumption was that any inconsistency was due to a parity error, not a data error. But with P and Q parity there's scope to use the parity check information more wisely. Has that actually been implemented?

 

Running a correcting parity check will always correct any sync error, be it P, Q or both.

 

My monthly checks are always non-correcting, if I get an error, which I usually don't, I then decide how to proceed.

Link to comment

As it's the first of the month and I have a scheduled parity check underway with "Write corrections to parity disk" set to the default "Yes" in the scheduler, I was wondering if it will do literally that - now that I have dual parity - or will it treat any error in a more intelligent way? With only one parity disk the assumption was that any inconsistency was due to a parity error, not a data error. But with P and Q parity there's scope to use the parity check information more wisely. Has that actually been implemented?

 

Running a correcting parity check will always correct any sync error, be it P, Q or both.

 

My monthly checks are always non-correcting, if I get an error, which I usually don't, I then decide how to proceed.

 

I think what John_M is saying is this:

 

With one parity disk unRAID can detect a parity error, but it doesn't know for sure which disk the error is on so it can just "fix" it as it assumes the error is on the parity drive and not bit-rot.

 

With two parity disks it should be possible to pinpoint which disk has the error and correct it.  So if you run a parity check with two parity drives unRAID "could" find and fix bitrot.

Link to comment

 

I think what John_M is saying is this:

 

With one parity disk unRAID can detect a parity error, but it doesn't know for sure which disk the error is on so it can just "fix" it as it assumes the error is on the parity drive and not bit-rot.

 

With two parity disks it should be possible to pinpoint which disk has the error and correct it.  So if you run a parity check with two parity drives unRAID "could" find and fix bitrot.

 

It's not possible to pinpoint the disk, at least not for now, but with dual parity, if both P and Q parity are wrong, the error is on one of the disks, but you can't know which one.

 

If there is a P or Q error only then it's on the parity disk.

Link to comment

 

I think what John_M is saying is this:

 

With one parity disk unRAID can detect a parity error, but it doesn't know for sure which disk the error is on so it can just "fix" it as it assumes the error is on the parity drive and not bit-rot.

 

With two parity disks it should be possible to pinpoint which disk has the error and correct it.  So if you run a parity check with two parity drives unRAID "could" find and fix bitrot.

 

It's not possible to pinpoint the disk, at least not for now, but with dual parity, if both P and Q parity are wrong, the error is on one of the disks, but you can't know which one.

 

If there is a P or Q error only then it's on the parity disk.

 

Thats made it clearer for me! Thanks for replying!  :)

Link to comment

I was asked for SMART values. One can see in the logfiles that I set the SMART configuration correctly for my controller card. I can see all SMART values in the WebUI, can do tests and see all raw values. When diagnostics do not export the values though I configured the SMART correctly for every disk, this is obviously a bug in the diagnostics software that does not honor the SMART configuration.

 

Can you test the attached diagnostics script? It includes now the SMART configuration parameters. Thanks.

diagnostics.zip

Link to comment

As it's the first of the month and I have a scheduled parity check underway with "Write corrections to parity disk" set to the default "Yes" in the scheduler, I was wondering if it will do literally that - now that I have dual parity - or will it treat any error in a more intelligent way? With only one parity disk the assumption was that any inconsistency was due to a parity error, not a data error. But with P and Q parity there's scope to use the parity check information more wisely. Has that actually been implemented?

 

Running a correcting parity check will always correct any sync error, be it P, Q or both.

 

My monthly checks are always non-correcting, if I get an error, which I usually don't, I then decide how to proceed.

 

Yes, I realise that with only two bits to play with the scope is limited but I don't like the assumption that if there's a disagreement with both P and Q then they must both be wrong. It's surely more likely that a data disk is in error in that case, but which one is still unknown. So what to do in that situation?

 

With one parity bit there can only be two situations - either all is good (assuming there isn't a double error at the same location) or there's an error, in which case it's assumed that the parity is wrong.

 

With two parity bits there can be four situations - all good, P is bad, Q is bad or a data disk is bad. Again, assuming there is not a double error at the same location.

 

I'm going to change my monthly parity check to non-correcting. Thanks for the reply.

 

Link to comment

...If there is a P or Q error only then it's on the parity disk.

I wonder if this is really a valid assumption. The only time I have ever had parity errors, and something that always triggers a correcting parity check, is the so-called unsafe shutdown. If I understand correctly, in the case of single parity and unsafe shutdown, it is assumed that the data was written but parity didn't get written before power was cut, so parity is in error. In the case of dual parity and unsafe shutdown, couldn't it also be the case that data was written but neither parity got written before power was cut?
Link to comment

...If there is a P or Q error only then it's on the parity disk.

I wonder if this is really a valid assumption. The only time I have ever had parity errors, and something that always triggers a correcting parity check, is the so-called unsafe shutdown. If I understand correctly, in the case of single parity and unsafe shutdown, it is assumed that the data was written but parity didn't get written before power was cut, so parity is in error. In the case of dual parity and unsafe shutdown, couldn't it also be the case that data was written but neither parity got written before power was cut?

 

Unsafe shutdown sync errors are usually in the beginning, and the user knows it should expect them, but I agree, in this case, both paritys will be wrong and should be corrected.

 

I don't remember the last time I got an unexpected sync error on any of my servers, but except the above situation, PQ errors are probably a disk error, like from reallocated sectors or other disk issues and IMO should be investigated before correcting parity.

 

 

Link to comment
Guest
This topic is now closed to further replies.