Help, in working towards upgrading to V5 from V4.3.3 I was upgrading to the first step of V4.7

I copied the two bz files and memtest and rebooted the server (15 drives + Parity with no drive changes) and V4.7 installed.

 

But the array didn't start because it reports my disk4 (SDO 1TB) as red because "my replacement disk is to small"

No disk was altered or changed.  Just upgrade to V4.7 from V4.3.3.  I do notice that the total overall size is 4 blocks less under V4.7   V4.3.3 see's it as 976,761,496 and V4.7 see's it as 976,761,492

 

I don't see any HFA issues in the syslog for this drive. Just a "Wrong" line added.

Before the upgrade the array was all happy with no errors.  If I revert back to V4.3.3 the array starts.

 

Best solution/steps to get this working ?

Thanks for any help.

 

SOLUTION:

Installed directly to V5.0.5 bypassing the recommended install to V4.7 first before going on to V5.0.5.

V4.7 showed 4 blocks less on one drive so array would not start.  However V5.0.5 showed the original 4.3.3 reported size.

 

Steps:

Saved my old 4.3.3 disk mappings and remapped them under V5.0.5 except the parity drive.

Checked that all drives didn't have any MBR issues before starting the array.

Started the array (without a parity drive!)

Run the UTIL "New Permissions"

Stopped the array

Added my parity drive (new 4TB from old 2TB drive)

Let parity get created on new drive.

All happy 15 green drives + parity.

Syslog.txt

4.3.3? Holy cow... I can't imagine how old that must be.

 

Anyways... I don't know what (if any) plugins you are running. Personally I would skip the multi-hop approach and do a clean install of 5.0.5 and just reassign the disks. Just make sure you have a screenshot of your original config so you know which drive to assign to parity and cache.

 

You can backup your existing config, re-format your USB key, drop in 5.0.5 and your license key and start up UnRAID. Then re-assign your drives. If you trust your parity I think you can choose the option that parity is valid before you start, and then test parity once you are up and running.

 

You can always revert back to 4.3.3 if needed with your backup.

 

I am going to point out I've never used anything that old... I started on the 5.0 betas, but don't see any reason this shouldn't work for you. It will also give you a nice clean config to start from.

 

You will want some of the details out of your disk.cfg network.cfg and ident.cfg as well. Chances are you can just copy them into your new config directory with the license key as I doubt anything has changed, but again... you may want to just re-config UnRAID with the same settings.

 

Others may have suggestions to get 4.7 "fixed" for you, but if you don't have a lot of config I think a clean 5.0.5 install makes more sense, but it may just be me.

The readme for the latest stable version says to get to V4.7 first from V4.3.3 ?  Ok to jump directly to v5? 

 

When I build this array it was back in V3 days in 2008! This is it's only 2nd upgrade.  I'm a if it's not broke type of person and doing what you need leave it be.

 

The readme for the latest stable version says to get to V4.7 first from V4.3.3 ?  Ok to jump directly to v5? 

 

When I build this array it was back in V3 days in 2008! This is it's only 2nd upgrade.  I'm a if it's not broke type of person and doing what you need leave it be.

 

The slow, happy, safe approach is to upgrade to 4.7, then to 5.0. What I am suggesting is a wipe and rebuild vs an in-place upgrade.

 

Basically you are taking a Windows 98 machine, upgrading it to Windows XP and then upgrading to Windows 7. I am saying, wipe out the USB key and do a clean Windows 7 install.

 

Just make sure you have a good backup of your existing USB key first, and the screenshot of your disk assignments. All you really care about is that the parity disk and cache disk (if used) are in the correct positions. With 5.0 the data disk assignments don't matter, but you can choose the same order you had in 4.3 if it makes you more comfortable.

 

Help, in working towards upgrading to V5 from V4.3.3 I was upgrading to the first step of V4.7

I copied the two bz files and memtest and rebooted the server (15 drives + Parity with no drive changes) and V4.7 installed.

 

But the array didn't start because it reports my disk4 (SDO 1TB) as red because "my replacement disk is to small"

No disk was altered or changed.  Just upgrade to V4.7 from V4.3.3.  I do notice that the total overall size is 4 bytes less under V4.7   V4.3.3 see's it as 976,761,496 and V4.7 see's it as 976,761,492

 

I don't see any HFA issues in the syslog either. Just a "Wrong" line added.

Before the upgrade the array was all happy with no errors.  If I revert back to V4.3.3 the array starts.

 

Best solution/steps to get this working ?

Thanks for any help.

 

If you go back to 4.3.3 is all ok?

 

I have been using unRAID since 4.2.1 (I think). I do not remember a situation when drive size changed from version to version. But my memory isn't perfect and there might have been something I am forgetting. The history of the forum is complete back to that time so you could do some searching to confirm.

 

One of the reasons not to wait so very long to upgrade, though, is the people forget the details after several years.

 

I would suggest an email to Tom. He may suggest recreating the key, defining a "new config", and rebuilding parity. But I am concerned that if the drive is reporting a different size, that the data on that disk may be improperly aligned or something, and may not mount properly. If so, that data is in danger of being lost. Again, Tom would be the best one to contact with something like this.

If you go back to 4.3.3 is all ok?

Yes, going back to 4.3.3 and everything is fine and the array starts.

I wouldn't think it would be off sector wise since it's just a change of four bytes!

 

Emailed Tom but haven't heard back if it's ok to jump to 5.0.x from 4.3.3 directly.

If you go back to 4.3.3 is all ok?

Yes, going back to 4.3.3 and everything is fine and the array starts.

I wouldn't think it would be off sector wise since it's just a change of four bytes!

 

Emailed Tom but haven't heard back if it's ok to jump to 5.0.x from 4.3.3 directly.

 

Stick with 4.3.3 until we figure out why this is happening. I've directed Tom here to take a look. Hopefully he'll see my note or your email and chime in. Sit tight til then.

When did you last run a parity check on this system?    ... and was it error-free?  (Zero sync errors and no disk errors shown)

 

I vaguely remember this size issue being a "hiccup" along the way to v4.7, but it's been too long for my ageing memory to remember exactly what the resolution was ...

 

... but not so long ago that a quick search through the forum couldn't find it 

 

Here's the same issue back when v4.7 was released:  http://lime-technology.com/forum/index.php?topic=10483.msg99685#msg99685

 

... and the very next post by JoeL provides the fix:  http://lime-technology.com/forum/index.php?topic=10483.msg99686#msg99686

 

In your case it would be hdparm -N p976761496  /dev/sdf

 

 

 

However, rather than a step-wise upgrade process, I'd just do as bkastner suggested ... back up your flash drive -- be CERTAIN you have a copy of your key file; and that you KNOW which disks are assigned to each slot (the only one that actually matters is the parity drive).

 

Then just reformat the flash drive; load it with a clean copy of v5.0.5;  add your key file to the Config folder; and then boot to that version.    Go to the Web GUI and assign all your data drives (NOT your parity drive) ... and then confirm all is okay and you can access the array okay.    Then Stop it; assign parity; and restart it and let the parity sync run.    When it's done, do a parity check to confirm all is okay; and then you're ready to configure any of your add-ins/plug-ins, perhaps add UnMenu, etc.

 

Note that you do NOT want to do this if you KNOW that you actually have a failed drive in the system ... but if you had a "clean" parity check and no red-balls before your upgrade, then it's a quick and simple way to ensure you're at the latest version.

 

Before on 4.3.3 it was all green and no parity errors.  Last done around 1 week before I started the upgrade.  I'll wait to hear from Tom before jumping to 5.0.5

I'll wait to hear from Tom before jumping to 5.0.5

 

Good idea ... the two most knowledgeable folks on this issue are Tom and JoeL

 

Before on 4.3.3 it was all green and no parity errors.  Last done around 1 week before I started the upgrade.  I'll wait to hear from Tom before jumping to 5.0.5

 

I do not think this is an HPA issue. I would wait to hear from Tom.

Agree ... as I noted above, the best folks to comment on this are Tom or Joe

 

But a few more thoughts ...

 

There were quite a few folks who had a size issue when migrating from 4.x (where x < 7) to 4.7 -- the thread I referenced above has at least a dozen examples of that (I quit counting/reading after 15 pages or so).    Note that several of these had the same 4 block differential that you're seeing ... with fixes that weren't always identical.   

 

Tom made a couple posts on this on page 10 in the thread ... basically just noting that he was considering ways to alter the handling of drives with this issue -- but there was never a follow-up release in the v4 series.    Most of the posts advising folks how to resolve the issue were from JoeL.

 

If you want to keep all your current configuration data, wait until you get the guidance you need to successfully upgrade to v4.7 (or carefully follow the guidance that was provided 3 years ago in that thread).  But note that a lot of the older plugins/addons aren't going to work with v5 anyway.

 

By the way, I just noticed (in reviewing the thread) that you said your drives were only different by 4 bytes.  That's not correct -- they're different by 4 blocks.

 

AND this IS almost certainly an HPA issue.    The drive sizes in question on the troublesome drive are 976,761,496 (shown in 4.3) and 976,761,492 (shown in 4.7 as "too small").  Note that the CORRECT size for this drive without an HPA is 976,762,552

 

 

Personally, I'd just upgrade directly to v5.0.5 by starting with a fresh install of UnRAID, as bkastner and I suggested earlier.    If you're concerned about whether or not this may cause issues with your current data, there's a simple way to safely check this ==>  Create a new USB flash drive with a clean v5.0.5 basic (free) install.    Boot your system with that flash drive.  Then add the troublesome drive (SN S13PJ1FQ300810) as the ONLY data drive ... with no parity.    Confirm that it doesn't show that drive as "Unformatted" (Do NOT Start the array if it does).    Then just Start the array, and you can then access the drive from a PC to confirm the data is all there.  Do NOT do any writes to it, so it's not modified by this test.    As long as everything's okay with that, you can safely do as outlined earlier to just move directly to v5.0.5.    There is NO reason to remove the HPA from the drive ... the amount of missing space is insignificant.

 

 

Ok - went back to twilight's syslog.

 

Sure enough, there is an HPA on disk4. (sorry gary, how could I have doubted you  )

 

Still doesn't make sense to me. If there is an HPA, then I would think that all versions of unRAID would see the disk as the smaller size, and if the array was created when the disk was larger, report a size error - be it 4.3, 4.7 5.0.5, or beyond. But maybe 4.7 put in an additional check that was not in place for prior versions. Not sure. I do not think jumping to a later version (e.g., 5.0.5) would fix this problem. The HPA would remain.

 

Quick digression - What is an HPA? An HPA is a small amount of disk space that is reserved on the disk, effectively making the disk smaller. The extra space is typically used for a very special purpose (for example keeping a backing of the BIOS settings). An HPA will not cause any harm if it was created BEFORE the disk is used in the array. What has happened (apparently) is that the disk was added to the array when it was full sized, and AFTERWARDS the HPA was added. Since the HPA effectively reduces the size of the disk, unRAID now thinks the disk is too small and it complains. In such a case you might think that you could just rebuild onto the disk leaving the HPA in place - but the problem is the disk is a tiny bit too small and unRAID will not rebuild onto a smaller disk. So the solution is to get a new disk or remove the HPA so that unRAID will see the disk as sufficiently large and allow you to rebuild on it.

 

Twilight - are you using a gigabyte motherboard, or did you use a gigabyte motherboard with this disk at some point in the past?

 

If you are currently using one, you need to do some research and disable the setting that causes the BIOS to create the HPA. Otherwise, removing it will just result in it being created again (or possibly worse cause it to happen on a different disk, which would be very serious in the current situation). Until this is done do not try to remove the HPA. If the disk used to be on a Gigabyte MB, but you have upgraded, the HPA will not be recreated and it will be safe to just remove it. Realize that once an HPA is created it is created and will not go away without it being manually deleted. The goal is to delete it and not have it recreated by the motherboard.

 

Another digression - you might think that telling the BIOS to not create the HPA is sufficient, but it is not. If the BIOS defaults to having the HPA created, then if the battery dies or you ever tell the motherboard to load default settings, the setting will be reverted to CREATING it again, and poof, fast as you can reboot, an HPA will be created. I believe that Gigabyte updated the BIOS of at least some motherboard such that this is NOT the default. This is really what you want as your long term solution.

 

Doing the following will not remove the HPA and can be safely done while you investigate. But you need to turn off the HPA creation before doing the actual removal - for which I will post instructions after you do the following.

 

Run the following command (boot into 4.7 first)

 

  hdparm -N /dev/sdo

 

Post the output and await further instructions.

Relooking at gary and bkastner's comment on moving to version 5 - I do think that what they list is possible. The ultimate fix might be to assign ALL of the disks (including parity) and say to trust parity. Check all disks are readable. Do a non-correcting parity check and see if parity is good. It might just work without having to remove the HPA, although I would not be surprised if the disk4 would have a slightly corrupted RFS file system. You can stick with the 4.7 upgrade approach that is tried and true, but TBH, this is probably what I would do at this point as I don't think it would hurt anything so long as the array was not not written to.

 

If you do this, make sure you have a backup of the config folder to restore if you want to go back to 4.3.

 

 

The ultimate fix might be to assign ALL of the disks (including parity) and say to trust parity.

 

NO !!  Well ... at least not initially.

 

The FIRST thing is to assign all of the DATA disks ... but NOT a parity disk ... and confirm they all show okay (no "Unformatted" disk).    Then Start the array ... just as a confidence check ... and read some of your data from one of your PC's so you're confident you've got v5.0.5 running okay.

 

NOW you can Stop the array, assign parity, and restartit; OR, if you want to skip the initial parity sync, you can do as Brian (bjp999) suggested:  Do a New Config (on the Utils tab), and re-assign all the data disks, assign the parity disk, and check the "Trust Parity" box ... this will skip the initial parity sync, and use the parity as it currently is.      If you do this, I would then run a correcting parity check, just in case the HPA-related area is treated differently than it was in v4.3 (since v4.7 does this, newer versions may as well).    If that's the case, there will be a few sync errors to fix.    When that completes, if all is well (zero sync errors, no disk errors) you're good to go.  If there were in fact sync errors, then run it again -- this time it should be clean.    As I noted before, there is NO reason to make any attempt to remove the HPA -- there's simply not enough "lost" space to be worth the risk to your data.

 

 

Another note:  If you're paranoid about the possibility of data loss from the "troublesome" drive, just boot to v4.3, and copy all of the data from disk4 to a backup location on another PC.    You should, of course, have backups anyway, but if you don't at least backup this one drive [You are NOT going to lose data from it by moving to v5.0.5, but this would give you further peace of mind].

 

...  You could also just do the little "test" I outlined a couple posts ago (with a new flash drive and ONLY that disk assigned as a data disk -- to confirm v5.0.5 "sees" it okay)

 

... or you could attach this drive to a PC; install the free Linux Reader [ http://www.diskinternals.com/linux-reader/ ] ;  and confirm you can read all of your data from it (Just copy a couple files to your PC's disk to confirm it's reading everything okay).    THEN you could update your flash drive to v5.0.5 (as outlined earlier);  and assign all of your OTHER data drives (but NOT this drive).  Confirm all is okay; add the parity drive; do the parity sync; then run a parity test to confirm it ... and THEN you can simply copy all of the data from the "troublesome drive" (with the HPA) to your array over the network ... and you'll no longer have a drive with an HPA in your array      If your array is too full to do that, just add another disk to it, or, if one of your purposes for this upgrade is to get > 2TB support, add a larger drive as the parity drive (I'd just do this while upgrading -- no need to do a parity sync to the old, smaller parity drive if you're going to immediately replace it), and use the old parity drive as another data drive.

 

Still doesn't make sense to me. If there is an HPA, then I would think that all versions of unRAID would see the disk as the smaller size, and if the array was created when the disk was larger, report a size error - be it 4.3, 4.7 5.0.5, or beyond. But maybe 4.7 put in an additional check that was not in place for prior versions.

 

I agree ... but clearly v4.7 made a change that caused this problem for a LOT of folks -- sometimes with a 4 block differential (as here);  other times with some other difference; but it caused a lot of headaches.  The thread I referenced earlier shows a LOT of examples of this -- and the hassle of resolving it  (which, if not done properly, could cause data loss).    There are 28 pages of comments in that thread, with at least a dozen cases of this issue ... and it's not the only thread that discussed this specific problem.

http://lime-technology.com/forum/index.php?topic=10483.msg99518#msg99518

 

That's why I think the simplest way to resolve this is to just do a "clean" install of v5.0.5 and not bother with removing the HPA.

 

 

I do not think jumping to a later version (e.g., 5.0.5) would fix this problem. The HPA would remain.

 

Yes, the HPA would remain.  But the simple fact is that's not an issue.    All it means is that the one disk with this "problem" would be slightly smaller than other disks of the same native size.    The only time an HPA is actually a real "problem" is if it happens to be on your parity disk.

 

 

I do NOT think there is ANY data loss risk by moving directly to v5.0.5, but if there are no backups available, and the OP wants to be absolutely certain of that, the steps I outlined in my previous post provide several ways to be confident of that.

 

By the way, I just noticed (in reviewing the thread) that you said your drives were only different by 4 bytes.  That's not correct -- they're different by 4 blocks.

 

AND this IS almost certainly an HPA issue.    The drive sizes in question on the troublesome drive are 976,761,496 (shown in 4.3) and 976,761,492 (shown in 4.7 as "too small").  Note that the CORRECT size for this drive without an HPA is 976,762,552

 

 

Personally, I'd just upgrade directly to v5.0.5 ......

 

Interesting, there is a HPA entry but it's for a different drive (2TB) drive that reports the same between 4.3 and 4.7.

 

The drive causing issues (#4 - SDO) was reporting less under 4.3 and 4 bytes less than that under 4.7 (I don't think it reports a HPA issue in the log for drive SDO).

 

I like the idea of making a 5.0.5 key and seeing if Disk4 is recoginized ok as a precheck.

The drive causing issues (#4 - SDO) was reporting less under 4.3 and 4 bytes less than that under 4.7 ...

 

Just to clarify ... as I noted earlier, it's not "4 bytes less" ... it's 4 thousand bytes less (4 1k blocks)

 

I like the idea of making a 5.0.5 key and seeing if Disk4 is recoginized ok as a precheck.

 

I'd simply do that, and after you confirm it's okay, just move directly to v5.05