TsA here, so first I want to say this community has been very helpful.

 

I finally got my dual headed, 4k gaming, NAS system online and stable. (for the most part)

2 - 8tb wd red, one in parity, one as a disk1 in the protected array. Formatted as xfs

2 - unassigned 500gb SSD (formatted NTFS direct passthrough to windows 10 pro VM)

2 - 8tb external drives for manual backups offsite.  One being a backup for the backup.

 

1. How does UnRaid physically write files to disks through a windows VM?
2. My question is, how can I divide my brand new array into physical sections, kind-of like physical partitions, so as I start building it I can manual fix some of the fragmentation issues with my old files. (i.e. 2TB longtermstorage, 4TB movies, 1TB sharedworkstation, 1TB cache)
3. Should I not be so worried about the physical fragmentation of the data. (I want to be prepared for a data recovery company if, god forbid, catastrophic failure)

Thank you,

TsA

Edited November 28, 20187 yr by tsawind
Added picture

Update on progress, woke up with morning and sytem was turned off, haven't found where to look to find any logs of why it turned off.  Machine had array running and 2 VMs of windows 10 pro running. Re-sources have been split 50-50 to VMs. (7gb ram, 4 cores 8 threads 1 500gb SSD). I suspect something with the VMs cause the crash, maybe I need to learn how to dedicate one core to unraid, 4 to primary VM, and 3 to second VM?

I started the array and now it is parity checking with the 2 8TB wd drive.  I was planning to start writing data, but now I am hesitant, concerned about stability and parity check will take like 12+- hours.
I am going to reboot into a baremetal, clean install of window 10 pro and start doing writing data onto one of the WD easystore 8tb NFTS external drives. I have maybe about 3 TB of data to try to recover and go through, about 500gb is what I would consider "critical".  (Wedding pictures, tax statements ect).
I'm hopeing this wont hurt anything to click cancel parity check, I really don't want this to be a normal thing or that parity drive really probably isn't for me.

Just now, tsawind said:

sytem was turned off,

Was it actually turned off or still on but crashed?  Turned off implies power problem.

Turned off, but It seems like it crashed over night.  I don't know how to check the proper logs.

unRaid's logs aren't stored anywhere but RAM, so aren't available after a reboot.

 

But, if it was turned off the first thing I'd suspect is a power failure and no UPS (or a UPS installed but the length of the power failure was longer than your UPS settings)

5 minutes ago, tsawind said:

Update on progress, woke up with morning and sytem was turned off, haven't found where to look to find any logs of why it turned off.  Machine had array running and 2 VMs of windows 10 pro running. Re-sources have been split 50-50 to VMs. (7gb ram, 4 cores 8 threads 1 500gb SSD). I suspect something with the VMs cause the crash, maybe I need to learn how to dedicate one core to unraid, 4 to primary VM, and 3 to second VM?

I started the array and now it is parity checking with the 2 8TB wd drive.  I was planning to start writing data, but now I am hesitant, concerned about stability and parity check will take like 12+- hours.
I am going to reboot into a baremetal, clean install of window 10 pro and start doing writing data onto one of the WD easystore 8tb NFTS external drives. I have maybe about 3 TB of data to try to recover and go through, about 500gb is what I would consider "critical".  (Wedding pictures, tax statements ect).
I'm hopeing this wont hurt anything to click cancel parity check, I really don't want this to be a normal thing or that parity drive really probably isn't for me.

The fact that when you started it back up, a parity check started means that there was an unclean shutdown. The parity check is to verify that the parity drive and the data drives are still "equal". Why would you not want not want to check that?

 

Thank you for the fast responses.

No I do not have an UPS yet, but I haven't had power failures at this house in months. (underground electric service)

And yes I understand parity checking from an unclean shutdown, that is a good thing.  The fact is there should have been no read/writes on the array between when I went to bed (parity valid) and woke up.  Also these are 2 brand new drives.  I mean Ill recheck parity that isn't an issue, issue is I don't want to start writing data during a parity check, I don't even know if unraid will let me do that sort of thing.

Will this hurt anything to cancel the parity check? I want to reboot to baremetal and start recovering data.
I will go buy a UPS and setup a video camera to monitor the screens I guess.  Any other suggestions while I have your attention?

Good to meet you squid, I have heard good things.

 

14 minutes ago, tsawind said:

I mean Ill recheck parity that isn't an issue, issue is I don't want to start writing data during a parity check, I don't even know if unraid will let me do that sort of thing.

You can always use the server normally during a parity check or rebuild.  Just reads / writes / parity check speed slows down.

 

So, did it come up as an unclean shutdown?  99.9% of the time on a crash, the computer will NOT shutdown.  If you didn't have a power failure, then I'd suggest a bad power supply, possibly CPU shutdown the system due to an overheat situation.  There is also a thread here from a few years ago that was determined to be a cat playing with the power switch.

 

8 minutes ago, tsawind said:

I have heard good things.

Don't believe a word of it.  Lies, more lies, and damned lies

11 minutes ago, tsawind said:

cancel the parity check?

Up to you and your own "peace of mind".  I can't suggest that you do that without incurring the wrath of people around here.

Edited November 28, 20187 yr by Squid

The cat thing is funny, because it actually was on my list of things to check.  I haven't re-covered (taped hard plastic over it) my power button on top of computer since the rebuild.  I am trying to recover data some 10yr old hard drives ect, very risky, which is why I am asking. I have not plugged drives into power in months (part of why I am building a NAS). I will just deal with unraid later, im canceling parity and rebooting to a clean windows 10 install on a SSD that I know is stable.  Ill check these forums later today.

Wish me luck that I don't have to start calling data recovery centers today!
 

Thank you,
TsA

Edited November 28, 20187 yr by tsawind

BTW figured I would mention my luck and how I got here.  Every time a order new hardware this happens to me.
 

Needed an upgrade, gaming rig was 10yrs old and still running strong.

Ordered a 1070ti
Bought 4 new sticks of DDR2 and some used fans
Installed a fan onto motherboard and shorted it out. (my fault)

Succession of deaths an suicides to follow:

1 motherboard murdered.

like 1 or 2 fans dead.

2 fans committed suicide in front of me. (both were 120mm)
1 Seagate 250gb drive committed suicide, just started clicking, no recovery.

1 Seagate 250gb drive spun up, read all data and copied onto 1.5tb WD green.  Format attempted and it committed suicide.

1 WD 750gb spun up, read almost all data (bad sectors) and copied onto 1.5tb WD green.  Format attempted and it committed suicide.
Other miscellaneous murders, suicides, and deaths from old age have been witnessed during rebuild.

Now I am lookin at this 1.5tb "DATE: 06 MAR 2010" drive that is full, a 500gb "Date: 10301" (I'm guessing full) that I haven't dared plug in, and a dozen USB flash drives and camera flash drives on my desk in front of me. I also have some really old misc. 80gb drives to check through and trash during this process.

I'd say I'm a bit nervous ATM and I believe that is the definition of Fragmented.  So an uknown, unclean shutdown, of a brand new NAS is a bit concerning to me ATM.

Wishing myself luck,
TsA

Update:
I believe I have some form of mental disability (like autism), just to put things into perspective.

I Just had an AHAH moment, Huge breakthrough, like party time.  I just learned about M-Disks.

I am ordering drives and disks to incorporate into the system, not really too concerned about extreme redundancy in the array if I can use M-Disks.  They theoretically will last 1,000+- years.  They come in 4.7 GB DVD, 25,50,and 100GB Blu-ray.  They are one time write and exactly what I have been looking for, for literally 50+ hours in the past month. And realistically been seeking this data medium for years.

Couldn't talk myself into turning off the parity check, it is at 25%.

M-DISC Is THE BEST solution I have come up with for redundancies. MAJOR MAJOR AHAH MOMENT.

Edited November 28, 20187 yr by tsawind

YAY Parity is 100% now, took all day....
Anyways I found out Excel Energy did have a power outage this morning.  First time in months.  I am not beginning to recover data.

What should I be looking for in a UPS, (minimum specs)

Thanks,
TsA

4 hours ago, tsawind said:

What should I be looking for in a UPS, (minimum specs)

apcupsd compatible, which means almost any APC branded unit, many Cyberpower work except for the turn off after shutdown option.

 

As far as capacity, that's up to your budget. Almost any except for the very smallest units will probably have enough battery to allow for a clean shutdown IF your windows 10 VM's cooperate with a commanded shutdown. If your VM decides to be stubborn and stay running, you could end up with a crashed VM when unraid forces the issue, or your batteries run out. The stock unraid UPS control software (apcupsd) has clients for windows as well, so you can install UPS monitoring in your VM's. When a power outage occurs, you can program your VM's to shut down after a few (2-5) minutes on battery, so when unraid shuts down your VM's are already off.

 

Your goal for maximum equipment longevity is to use the batteries to less than 50% discharge, which means maximum usable runtime of around 10 to 15 minutes for mid to upper level consumer UPS. Server grade UPS can add battery capacity for more runtime.

 

Do you have a method to measure the ACTUAL power draw of your system at full load? Idle power draw is useless, as everything spins up and works hard during shutdown.

2 hours ago, jonathanm said:

If your VM decides to be stubborn and stay running, you could end up with a crashed VM when unraid forces the issue, or your batteries run out.

Install the VM client software on the virtio iso disk and then tell unRaid to hibernate the VMs during a shutdown.  You'll never have a problem

1 hour ago, Squid said:

Install the VM client software on the virtio iso disk and then tell unRaid to hibernate the VMs during a shutdown.  You'll never have a problem

Does that initiate windows hibernation, or the KVM save / restore hibernation?

 

If it calls for windows to hibernate, I don't think windows will act on it if it's in the middle of an update install.

Windows Hibernation.   But, the odds are still far better that Windows is NOT in the middle of an auto restart / update sequence than if it's not enabled and then Windows will automatically install updates when turning off.

I don't care about the VMs, im talking about the array where the important stuff is.  (I dont want to have to do parity for 20 hours if the power spikes)  This is a 2 headed gaming setup, each head has its own SSD passthrough.  I made full backups for each install after I did all the quirks like turning off windows search, power setting ect ect.  These backups are saved on the UnRaid array, so if a VM starts to fail or get slow or ANYTHING, I just bleep boop bleep, reformat SSD, pre-setup "clean" install.  So as far as im concerned, if UPS hits 75% battery from 100% it can force close the VMs, keeping the router and UnRaid array online as long as possible.


Looking for at least 30 minutes power for UnRaid before it shuts down cluster.
Do I just need a wall plug from amazon that detects watts drawn by system? Then calculate largest number drawn during a stress test then divide by 30 minutes I assume?

Sounds like apcupsd compatible (whatever that is) is a requirement.
Buy a name brand UPS.

Ideally system will restart and array will online when power comes back on.

 

Thanks, TsA

If you are looking for 30mminutes runtime on UPS then you should probably size the UPS to handle at least twice that.    This is to allow for the fact that batteries degrade and also to give time for a tidy shutdown if that limit is reached.    If you are measuring the power draw as part of trying to size the UPS make sure that you have all your drives spun up at the time as that is the worst case.

Thank you for the Suggestions!! Ill post what I put into my amazon cart before I buy it.