How-To: Migrate from unRAID 4.7 to unRAID 5.0

[stchas]

I recently migrated my production server from unRAID Pro 4.7 to unRAID 5.0-rc10. While I had performed limited testing on a small test server and was confident that the latest release candidates were "ready for prime time," I learned a number of lessons that I felt were important to share with the unRAID community.  Here is a link to the Wiki article with steps to follow to help make the migration experience more predictable and less stressful:

http://lime-technology.com/wiki/index.php/Migrating_from_unRAID_4.7_to_unRAID_5.0

 

As always, please provide feedback here (and feel free update the Wiki) so all can benefit.

 

Kevin

 

[RobJ]

Really really great work!  I think it is a terrific idea, well implemented, and I learned some things myself from it.  I suspect there are a number of users that wish they had found it before now!  It certainly would have avoided the need for some of the support threads.

 

One more section that would be useful, is to touch on the issues a few users have had.  For example, a few users could not boot, needed to run the make_bootable batch file.  And a few users have noticed slow writes, need a memory limiting parameter to restore write speed.

 

Hope to see a lot more from you!!!

 

(more comments later, but no time at the moment...)

[denizen]

Thanks for putting together this excellent guide.

 

I have a question about section 8.2. :Migrate to cache only share model.

In this section you suggest setting Included disk(s) to cache.  On my system, even though I have a cache disk, I am not given the cache disk as an option to include.  It only allows me to include my "disks".

 

I am trying to troubleshoot some other issues and am wondering if my server is somehow set up wrong such that I am not able include the cache disk.

 

Attached is a screenshot.

 

Thanks

[stchas]

In this section you suggest setting Included disk(s) to cache.  On my system, even though I have a cache disk, I am not given the cache disk as an option to include.  It only allows me to include my "disks".

 

denizen,

 

I don't recognize this screen shot.  Are you using some other web interface besides the one supplied by LimeTech?

 

I suspect you need to find the equivalent to the unRAID Settings / Share Settings page and make sure the cache drive appears as one of the included disks.  Then I suspect you will be offered the cache drive in your checkbox-selectable list.  See the Setting up disk shares and user shares section of the wiki to see how I set mine up.

 

Kevin

 

 

[limetech]

Very nice!  Thank you for doing this.  Here are few corrections:

 

Setting up disk shares and user shares

 

    Include each disk that is a member of a disk share or user share in the "Included disk(s):" textbox on the unRAID Settings/Share Settings page (then press Apply)

        That includes flash and cache if they will be shared

    My "Included disk(s):" list looks like this:

 

disk1,disk6,disk7,disk8,disk9,disk10,flash,cache

* Alternatively you can leave this box blank to include all disks.

 

 

The 'Included disks' and 'Excluded disks' only apply to array disks, that is, disk1, disk2 ... disk24.  The terms 'cache' and 'flash' are ignored and not used.  Also you can use a shorthand notation for a consecutive sequences of disks, so your 'Included disks' mask could be entered like this:

 

  disk1,disk6-10

 

---

I don't think there are any settings changes that require a reboot.  Most changes take effect "immediately", some may require array Stop/Start.  Might be some plugins that require reboot, but most should not.

---

For a cache-only share it is only necessary to select 'Use cache disk: Only' when creating the share.  'Allocation method' doesn't matter, and all other fields can be left blank.  Note that an existing share with using at least one disk in the array can not be "converted" to a cache-only share (unless you do it manually).

[jbartlett]

Using a text editor

 

I would highly recommend that you mention that the text editor needs to be able to support Unix style files where they only use Line Feeds.

 

Using Windows editors such as Notepad & WordPad would put carriage return characters in the file.

 

Unix style text files are easy to spot with Notepad - all the lines are merged with a square where the line feeds would normally go. Wordpad auto-translates them.

[RobJ]

Using a text editor

 

I would highly recommend that you mention that the text editor needs to be able to support Unix style files where they only use Line Feeds.

 

Using Windows editors such as Notepad & WordPad would put carriage return characters in the file.

 

Unix style text files are easy to spot with Notepad - all the lines are merged with a square where the line feeds would normally go. Wordpad auto-translates them.

 

Good points, and I added a link to the Editors section of the UnRAID Topical Index.

 

However in this one instance, Tom 'designed' the UnRAID go file to be flexible, because many new users would be Windows users.  See Is "go" in UNIX format?

[limetech]

Using a text editor

 

I would highly recommend that you mention that the text editor needs to be able to support Unix style files where they only use Line Feeds.

 

Using Windows editors such as Notepad & WordPad would put carriage return characters in the file.

 

Unix style text files are easy to spot with Notepad - all the lines are merged with a square where the line feeds would normally go. Wordpad auto-translates them.

 

Good points, and I added a link to the Editors section of the UnRAID Topical Index.

 

However in this one instance, Tom 'designed' the UnRAID go file to be flexible, because many new users would be Windows users.  See Is "go" in UNIX format?

Right, all unRaid-specific configuration files (ending with .cfg as well as the 'go' file) are first stripped of any 'carriage returns' before being sourced.

[stchas]

Thanks jbartlett, RobJ, and limetech for your comments.  I've updated the wiki to capture your insights.

 

Kevin

[reggie14]

I'm still confused about the need for the powerdown script (and have been ever since I started using unRAID).  Do I still need to install some sort of customization so I can use the power button on my unRAID box safely?  It doesn't look like the powerdown script thread has been updated for a while, and I don't know what that means.

[RobJ]

I believe we need to add more detail about the new permissions and security model.  Not sure who wants to flesh it out better, but would be best if it is someone who has actually converted their shares to the new stuff (I haven't used any of it).  There are now more ways to set up shares, but some (many?) users want to keep some or all shares available without any logins or passwords.  Another issue, some users have problems with Windows stations that need credentials deleted.

 

Some links for reference:

* For general help on the new permissions/security setup, see http://lime-technology.com/wiki/index.php/Plugin/webGui/SMB

* For more info on the new security model (a little old, may no longer be accurate?), see http://lime-technology.com/forum/index.php?topic=7047.0

* As to being able to avoid requiring passwords, see Tom's comments here

* And here's an example of a user encountering problems when upgrading

 

By the way, there is some newer UnRAID documentation (some is older) -> http://lime-technology.com/wiki/index.php/Plugin/webGui.

[stchas]

I have some info on permissions and migrating from go-based apps to their plugin versions that I'll be adding over the weekend.  I'm out of town this week so can't access my unRAID to validate the details.

 

I don't use any of the security or users features, so I'll ask others familiar with their setup to contribute.

 

Kevin

[Lacehim]

Great idea.  I'm sure many users will use this before upgrading.  I wish it had been available when I upgraded!  It would have saved me a heap of time searching the forum, especially regarding permissions on an apps drive/cache.

 

I vote that this becomes a sticky!

[stchas]

I'm still confused about the need for the powerdown script (and have been ever since I started using unRAID).  Do I still need to install some sort of customization so I can use the power button on my unRAID box safely?  It doesn't look like the powerdown script thread has been updated for a while, and I don't know what that means.

 

The apcupsd-3.14.10-i468-2atv.plg plugin installs

 

    apcupsd-3.14.10-i486-1_atv.tgz and

    powerdown-1.02-noarch-unRAID.tgz packages.

 

If you don't need the apcupsd package, you can find the powerdown package separately here.

 

The powerdown package also includes the

 

  /sbin/powerdown script that assists with a clean unRAID powerdown.

 

I have it installed on my system and have not implemented any "customization"

 

I did a test to verify that pressing the power button does shut down my unRAID box.  In the interest of full disclosure, I did it when my array was stopped (since this is my production server).  Be advised, however, that different systems have different responses.  A good read through Joe L's discussion on the topic here might give you some additional insights.

 

Kevin

 

 

[stchas]

I added a section on backing up your apps on the cache drive and some troubleshooting lessons learned with Crashplan on 5.0 to the wiki.

 

Kevin

[Frank1940]

Sure wish someone would make this a sticky!!!! 

[Joe L.]

unRAID will NEVER terminate processes using disks if you attempt to stop the array using its web interface.

 

If written correctly, and using the event handler in the latter 5.0 series, plugins can terminate their affiliated processes, HOWEVER... it is single threaded and any plugin can keep the process from completing if it is blocked waiting for something...

 

The supplied "powerdown" script simply presses the "Powerdown" button on the web interface.  It emhttp is blocked, or killed, it will do absolutely nothing.

 

In my personal opinion, the add-on powerdown script is necessary.  It has absolutely no dependency on emhttp, and will send kill signals to processes having open files keeping the array from cleanly stopping. 

 

Joe L.

[hwilker]

I have just used this guide to successfully upgrade from 4.7 to rc11. It has been a great help in getting the basic system up and running.

 

However, having got the basic system to run, , I am at a loss regarding the packages that I used to have installed on v4.7.

 

The following two lines were in my 4.7 go file.

 

/boot/unmenu/uu

cd /boot/packages && find . -name '*.auto_install' -type f -print | sort | xargs -n1 sh -c

 

I removed them, per the guide.

 

I'm unclear if I should add them back in to the new go file now that everything is running. Is unmenu even relevant in version 5?

 

Will including the next line "cd /boot/packages && find . -name '*.auto_install' -type f -print | sort | xargs -n1 sh -c" invoke all my old installed packages and set them to reload on boot? Are there packages that shouldn't be re-installed that would be if the second line above were included in the go file, if so, which ones.

 

Apcupsd seems to now be available as a plug-in. Does that mean I should delete the app from my packages folder?

 

Several other packages that seem rather essential (monthly parity check, pwerdown, and screen), or desirable (ssmtp, unraid-status-email) are in the packages folder, but I'm unclear if they are installed already or if including that line re-installs them.

 

Other package were listed under 4.7 as "Installed, Not Downloaded" such as hdparm, reiserfprogs, rsync and smartmontools). I' m unlcear if these are needed any longer or are now part of the system, and if needed does replacing that line in the go file re-install them as well.

 

I'm sure all this is obvious to a veteran linux user, but for someone with just a Windows background, this part of the process seems both scary and opaque.

 

Can anyone provide some guidance on how to proceed. I'd hate to screw up an upgrade process that has gone smoothly so far, by experimenting at this point.

 

Thanks.

 

Harry

 

 

 

 

 

[mbryanr]

You can continue to utilize the excellent unMENU (and/or SimpleFeatures) in 5.0, without those lines in your go script..they will not be reinstalled upon each reboot. None of the packages or plugins you mentioned are in the stock gui.

 

http://lime-technology.com/forum/index.php?topic=19510.0

 

I would be careful with duplicating packages/plugins. Particularly ones that use different versions of the same dependencies/libraries. But that is true for any package or plugin regardless if you are using unMENU or SimpleFeatures.

 

 

If you find you need smartmontools, hdparm, etc - then stay with unMENU. There is not a plugin manager (yet); which can cause problems with startup/shutdown depending on the plugin installed.  I have not had an issue with the basic SimpleFeatures plugins (I don't use webserver or sleep).

 

 

 

 

 

 

 

 

 

[hwilker]

You can continue to utilize the excellent unMENU (and/or SimpleFeatures) in 5.0, without those lines in your go script..they will not be reinstalled upon each reboot. None of the packages or plugins you mentioned are in the stock gui.

 

http://lime-technology.com/forum/index.php?topic=19510.0

 

I would be careful with duplicating packages/plugins. Particularly ones that use different versions of the same dependencies/libraries. But that is true for any package or plugin regardless if you are using unMENU or SimpleFeatures.

 

 

If you find you need smartmontools, hdparm, etc - then stay with unMENU. There is not a plugin manager (yet); which can cause problems with startup/shutdown depending on the plugin installed.  I have not had an issue with the basic SimpleFeatures plugins (I don't use webserver or sleep).

 

mbryanr,

 

 

Note: I have modified this post since, I figured out my own problem.

 

mbryanr,

 

I realized that I had been overthinking everything, due to a lack of understanding about the separate nature of the packages and the unRaid kernel. I think I've got it now. I installed the SimpleFeatures plugins but wound up removing them because they leave out a few key things I need - APC control, screen, and even though it has preclear I couldn't get it to show up.

 

I didn't understand that just re-including the following lines:

 

/boot/unmenu/uu

 

cd /boot/packages && find . -name '*.auto_install' -type f -print | sort | xargs -n1 sh -c

 

in my go script would re-install all the packages I had been using.

 

I've done that now and I'm back to an environment I understand. Basically the unmenu package with the packages I had and the new rc11 kernel.

 

I'll go with that as my main reason for upgrading was to enable the use of 3 and 4 TB drives, and the rest of the system seems to be working.

 

Just judging from myself, but I suspect a lot of users who are not normal Linux denizens, are getting confused in similar ways. Perhaps the very useful FAQ that started this thread could be updated to make a little clearer what the choices are after installed v5,  using the stripped down factory original go file.

 

Anyway, Thanks a lot for putting up with my ignorance.

 

 

Harry

[stchas]

Just judging from myself, but I suspect a lot of users who are not normal Linux denizens, are getting confused in similar ways. Perhaps the very useful FAQ that started this thread could be updated to make a little clearer what the choices are after installed v5,  using the stripped down factory original go file.

 

Good point, Harry.  I added starting unMENU as an example of an item you would have to restore to the go file in the wiki under Restore basic functionality utilities in the go file.

 

Kevin

[reggie14]

Has anyone installed and used APCUPSD recently.  It sounds like the plugin script doesn't work anymore, as the actual plugin isn't available on the Dropbox site anymore.  I haven't gotten terribly clear help on how to install it.

 

I'm also concerned that the last version, which hasn't been updated in a while, is rumored to have a bug in it, and I can't figure out how serious it is.

 

It's too bad UPS support still isn't built-in.  It seems like a pretty important thing for a file server like this.

[hwilker]

Has anyone installed and used APCUPSD recently.  It sounds like the plugin script doesn't work anymore, as the actual plugin isn't available on the Dropbox site anymore.  I haven't gotten terribly clear help on how to install it.

 

I'm also concerned that the last version, which hasn't been updated in a while, is rumored to have a bug in it, and I can't figure out how serious it is.

 

It's too bad UPS support still isn't built-in.  It seems like a pretty important thing for a file server like this.

 

I had the exact same problems yesterday. I couldn't figure out where to download the package. I wound up in a long thread here on the forum

 

http://lime-technology.com/forum/index.php?topic=1528.msg16563#msg16563

 

that at one point had a link to the package on someone's personal web site. I tried it. It didn't seem  to do anything. Ultimately I got unmenu installed and used the package that could be installed from the package manager in unmenu as I did under 4.7.

 

Re-inserting the following two lines in my go file:

 

/boot/unmenu/uu

 

cd /boot/packages && find . -name '*.auto_install' -type f -print | sort | xargs -n1 sh -c

 

both installed unmenu and all the packages that I had running under 4.7. Note that I didn't manually re-install anything, as with those two lines in the go file, they self-installed, presumably using the packages in the packages folder.

 

I don't know if this is what you were looking for, and I don't know what the syntax of the second line means but I just copied it from my go file under 4.7.

 

Harry

[reggie14]

I don't know if this is what you were looking for, and I don't know what the syntax of the second line means but I just copied it from my go file under 4.7.

 

Thanks, but its not exactly what I was looking for.  I don't want to install plugins I don't need, and I really don't think I'd need unMenu given what SimpleFeatures can do.

 

It's more than a little troubling to me that a simple question like this has been so difficult to answer.  I'm not really sure what it means, but it seems a little odd given that the community here is usually quite good at responding to these kinds of things.  I wonder how much of it is that everyone interested in moving to the v5-RCs has already done so.  So, if there are any new issues cropping up from new installs or new upgrades, maybe there's not a lot of people out there that can help.  I kind of wonder what's going to happen when v5 goes final, spurring a new round of unRAID users to upgrade.

[aaronwt]

Just for me to make sure, for me to go from 4.7 to rc12a. Do I really just need to know the order of the 22 drives I'm using, load rc12a and once I put the correct drives in the previous order I can bring the array up? Or do I need to copy over any files? I'm thinking about switching my main unRAID to 5.0 since I've been using it with my second unRAID I liek teh interface much better. But I have around 30TB of data that I need to make sure I won't lose.