You can indeed continue to use Reiser as your file system with v6.    Many prefer to move to XFS, but personally I did NOT do that -- primarily due to exactly what you've just discovered ... that Reiserfsck is an amazing tool for recovery should anything go wrong.    I happen to have always-current backups on a second server, so I COULD easily move to another file system, since if anything goes wrong I can simply copy from my backups ... but I just haven't seen any real reason to switch.    [At least not until we get some feedback on the recovery tools that work with XFS]

 

As for your "Lost & Found" folders ... there's no simple way to identify them.  You can try opening each file with different programs -- when you find the right one it should be clear what the file is.    But it can indeed be a tedious process.

 

I definitely agree with the suggestion to simply keep your older server and build a new one, so you'll have a complete set of backups.  You can THEN move the original server to v6 if you want.

 

While I appreciate Gary's view on the RFS vs XFS debate, I sort of wish he'd give both sides of the argument and let users make informed decisions on their own.

 

ReiserFS (RFS) is still supported, and reiserfsck has indeed proven itself a valuable tool. (Our experience with XFS has been small, although users have performed similar recovery exercises with XFS tools, but the nod still goes to the RFS tool.) Question is, is the terrific reiserfsck tool a reason to stick with RFS?

 

I think not. RFS has had a couple of serious defects, one of which went into unRAID in a 6.0 beta and a number of users have corrupted data caused by it (most not even knowing what files are impacted). RFS had already fallen out of favor generally with the Linux world, and these defects hastened others to abandon ship. (I think the defect is evidence that RFS is not a priority and not getting the level of support and testing of more popular filesystems like XFS.) You may not know it, but it's author is in jail for murdering his wife. After reading his story, I'd just as soon forget Hans Reiser and not advocate his namesake. He was in the middle of creating an updated file system (version 4) which his colleagues have struggled to complete.  Not sure the exact status, but we run version 3 in unRAID.

 

But murderer or not, by all measures, RFS was an good file system in its day. But it was created at a time when disks were measured in gigabytes not terabytes, and as the disks have gotten larger, many users (me included) have started having array hangs and timeouts caused by the slowness of the filesystem. Initially the problems showed up on very full disks, and was attributed to their fullness, but as disks got larger the slowness started when disks were not nearly as full. With 4T drives I was having issues when the disk went over 50-60%.

 

XFS is an extremely popular file system, used heavily in the commercial world as well as with hobbyists and personal users. It gets a lot of testing and attention with new Linux versions. Experience here has been that it does NOT slow down as disks get fuller. Switching to XFS has completely eliminated my disk-related performance issues and timeouts, and even the great and venerable reiserfsck is not going to send me back.

 

If you are using RFS and not having issues, there may not be a huge rush to move off of it. But you are in a continuously decreasing user base and are not going to get the level of testing with new Linux releases that XFS gets. You might consider plans to migrate, as a large number of experienced users here have. While I expect a lot of other unRAID users are still on RFS and are not making a conversion a priority, Gary may be the sole renaissance man advocating its a good idea to stay with it!

I'm not advocating RFS for NEW systems -- indeed, my v6 test system uses XFS ... and the new server I'm going to build this fall will as well.    My point is simply if you've got an RFS-based system I see no reason to migrate all of those disks to XFS.    ESPECIALLY those who don't have good backups and are depending on UnRAID's fault tolerance as their "backup"  (NOT, as you know, something I advocate doing -- a NAS is NOT a backup !!).

 

I've probably seen more data lost in the last year from folks who were trying to migrate all of their disks to XFS than for any other non-disk-failure related reason.  [usually because they encountered the user share issue]    I certainly agree NEW disks should be formatted with XFS in v6, but I simply don't think there's any compelling reason to change existing drives (especially those that are already full) ... and the experience in this thread with Reiserfsck is one reason why.    Focusing on putting a good backup strategy in place, and ensuring you've got a complete and current set of backups, is more important task than worrying about which file system you're using.

 

I agree RFS disks tend to get pretty slow for writes as they get close to full ... although I've not seen this happen anywhere near the 50-60% you indicate -- my experience has been more like 90% (including both my 3TB and my 4TB based servers).    All but 4 disks on my main media server are 98% full or fuller (8 of them show 100%) per MyMenu's display, and except for the last few DVDs written to them, there was never any major write issue.    Reading is, of course, not an issue.    Will I ever migrate those disks off of Reiser?  No.  Will I use Reiser on my new v6 server?  Also No.  When I ultimately retire the old media server, all of the data will have been copied to a new server using XFS -- but of course all of that data is also very well backed up.

 

 

 

Well, as always, thank you all for your meticulous advice as always. firstly, I think I have been somewhat remiss in the weeding area. The sheer relief of suddenly finding myself not as a ship dead in the water and files being recovered made me complacent on the first three drives with regard to the Lost + Found folder.

 

And I think I will revisit those drives perhaps running Reiser again, as i think I may have had a paradoxical view of what I believed to be rubbish.

 

The thought of another rig had not entered my head, but it has now  and I am filled with excitement. I presume that would be a standalone rig and just be Tower 2 on the windows box. (or would they be linked)?

 

My love affair RFS is basically born out of life saving,I would have had zero data without it, I can totally see the benefit of going forward with new supported systems. My dilemma is around the current rig, what is your advice around the way forward.

 

Is there any value in converting it? I am experiencing slow write speeds to the top end drives as Brian states, which I did not understand why before reading your updates. I would not want to place myself in a situation of possible data loss on the old rig that may not be recoverable.

 

As always I value all of your opinions, suggestions, and advice. Change and improvement are without question on the agenda, I just want to take the right steps with your help.

 

Peter

I'd focus on two things first ...

 

(1)  Finishing your data recovery

 

and

 

(2)  Building a new (2nd) server and copying ALL of your data to the new server.  On this server you can use v6 and XFS ... since you'll now have backups of your data, the recovery capabilities of RFS are much less important than they just were.    [i hate to think what your current situation might be if your drives had all been XFS ... although admittedly I don't know just how well the XFS recovery tools might have worked.]

 

You can do both of these at once if you want -- copying all recovered data to the 2nd server is certainly not a bad idea.

 

I presume that you'll want the newer server to become your primary unit, and the old server to then be relegated as your backup.    I would definitely upgrade the old server to v6 ... but NOT until you're done with your data recovery efforts.

 

The two servers will be independent -- you can name them whatever you want ... i.e. "Tower" and "Tower2";  "Main UnRAID" and "Backup UnRAID";  etc.    [Mine are called "Media", "Misc", and "NAS Backups"]

 

 

Well, a hick-up....

 

the server appeared to lockup, I have noticed a certain slowness since the rebuild of the drives, that was indicated by someone a few posts back. I wasn't to worried about that as the relief of data far outweighed that of time.

 

But today I seemed to have a total lockup I wasnt doing any of any consequence at the time so it was a mystery. I checked the sys log and there was nothing to suggest any problems.

 

I decided to safe power down and reboot, after the reboot things did still not appear as I would expect, my next move was to shut the array and restart it.

 

I then went to UnRaid main to restart it and.....................there it was the issue that clearly started all this, all the drives but one, the same one! shown as Unformatted !!!

 

Yes - I grabbed the sys log for you  I went through the process again and closed the array again, this a satisfactory position was revealed with the array showing shut and I restarted as normal.

 

If I was to make a guess here I would say the array is not shutting on the first attempt, if you had just put new drives in the situation would have a tendency to make you think it was just dealing with the new drives but clearly not the case!!

 

Foot Note..............Have ordered my new rig

 

Peter

syslog-2015-09-27.txt

Well, a hick-up....

...

... all the drives but one, the same one! shown as Unformatted !!!

 

Sounds more like a bad case of pneumonia than a hiccup !!

 

I'm wondering if your flash drive is having issues.    I'd get a new USB flash drive; install a fresh copy of UnRAID on it [i'd use v5.0.6];  and put assign TWO of your "unformatted" drives as data drives with NO parity drive ... and see if it shows the drives as unformatted [if so, do NOT let UnRAID format them;  if not, see if they appear to have the correct data on them.

 

IF that works (i.e. you can see the correct data; do a New Config and assign a different pair of your newly "unformatted" drives and see if they can also be seen.

 

If you can see the data okay on this new USB flash drive, then it's very likely your old one is corrupted and is likely the cause of your issues.  You'll need to contact LimeTech and ask that your key be transferred to the new flash drive  [Or you can put v6 on it along with your current key file and then you can do the replacement automatically through the Web GUI.]

 

 

... Have ordered my new rig

 

Good ... so in the not-too-distant future you'll be well backed up !! 

Well, a hick-up....

 

the server appeared to lockup, I have noticed a certain slowness since the rebuild of the drives, that was indicated by someone a few posts back. I wasn't to worried about that as the relief of data far outweighed that of time.

 

But today I seemed to have a total lockup I wasnt doing any of any consequence at the time so it was a mystery. I checked the sys log and there was nothing to suggest any problems.

 

I decided to safe power down and reboot, after the reboot things did still not appear as I would expect, my next move was to shut the array and restart it.

 

I then went to UnRaid main to restart it and.....................there it was the issue that clearly started all this, all the drives but one, the same one! shown as Unformatted !!!

 

Yes - I grabbed the sys log for you  I went through the process again and closed the array again, this a satisfactory position was revealed with the array showing shut and I restarted as normal.

 

If I was to make a guess here I would say the array is not shutting on the first attempt, if you had just put new drives in the situation would have a tendency to make you think it was just dealing with the new drives but clearly not the case!!

 

Foot Note..............Have ordered my new rig

 

Peter

 

This time, we do have errors, involving the SAS card subsystems.  There are none involving the flash drive so that should be fine.  Everything started fine, until the array began to start, then a kernel issue occurs involving the disk I/O (ATA) systems, followed immediately by the first failure of a GetIdentity request to a disk, followed slowly by the same failures to each port on the card, one by one, until all 7 had failed.  It retried one more time apparently unsuccessfully.  What's very weird is that the first time through, while one subsystem is failing to get identities, unRAID is mounting their file systems, apparently successfully it thought!  But probably not working correctly, as you noticed issues.  The thing about an identity request is, it contains not just the name and serial, but the physical parameters of the drive, most importantly the size of the drive and number of sectors.  If you can't get that, the sector count and size are zero, which makes all reads and writes fail, as their sector address is greater than zero.  This is probably the mechanism that caused the drives to look (wrongly) unformatted previously.

 

I can't tell if it's an issue with the SAS drivers and support modules, or the card itself.  You would probably learn which by upgrading to v6 and finding out if all problems are fixed or not.  As I noted before, I really don't trust that particular combination of SAS+SCST support for the SAS card, and that 4.7 was the last version it was ever used in.

Rob

Thanks for your reading of the sys log. It is my intention to put V6.xx on that rig when my new server arrives. Does this mean I need to make some hardware changes to to the rig before that?

 

Interestingly it does seem to sort itself out providing I close the array down again and start again, just a pity it caught me out the once and caused me all the issues.

Does this mean I need to make some hardware changes to to the rig before that?

I can't tell if it's an issue with the SAS drivers and support modules, or the card itself.  You would probably learn which by upgrading to v6 and finding out if all problems are fixed or not.

 

As I noted above, we can't tell if it's a hardware (the card) or software (the driver support) issue.  It would be interesting to see if a v6 upgrade resolved the issues, using the current rig.

I thought about if it was the right thing to add this to this existing thread, and decided I would like the advice of the of the people who saved me from disaster in the first instance. Particularly as I took on board their suggestions and acted accordingly.

 

Currently I have got a new server ( built on my behalf) which has the latest version of UR Pro which i was advised to do, the unit arrived and had problems which I tracked down 24Pin  mobo cable failure which took quite a long time to find.

 

As you will recall I have only experience with UR 4.7 my current dilemma is, in my efforts to explore the new server I appear to have lost access to my 4.7 box, the information recovered with your valuable assistance is safe, I just cant access it as windows does not see it.

 

I believe in my ignorant the new server has somehow over written the access to it the folder is empty and not reachable the box is currently powered up, so I am basically asking how I get that back.

 

I can I assume have more than one server on the same network? so I need to know how that is achieved or least get my old server back the intention being of course to update its UR.

 

Peter

I thought about if it was the right thing to add this to this existing thread, and decided I would like the advice of the of the people who saved me from disaster in the first instance. Particularly as I took on board their suggestions and acted accordingly.

 

Currently I have got a new server ( built on my behalf) which has the latest version of UR Pro which i was advised to do, the unit arrived and had problems which I tracked down 24Pin  mobo cable failure which took quite a long time to find.

 

As you will recall I have only experience with UR 4.7 my current dilemma is, in my efforts to explore the new server I appear to have lost access to my 4.7 box, the information recovered with your valuable assistance is safe, I just cant access it as windows does not see it.

 

I believe in my ignorant the new server has somehow over written the access to it the folder is empty and not reachable the box is currently powered up, so I am basically asking how I get that back.

 

I can I assume have more than one server on the same network? so I need to know how that is achieved or least get my old server back the intention being of course to update its UR.

 

Peter

Possibly IP collision. Are you sure the 2 servers have different IP addresses?

Could be IP collision OR could be a name conflict.

 

When you had the new server built, did you have them use a DIFFERENT name than your old server?  e.g. if they're both named "Tower" then it won't work if you have them both on at the same time.

 

If that's the case, do this:

 

(1)  Turn off your PC and both of your servers.

 

(2)  Boot the NEW UnRAID server and your PC.    Go to the web GUI, then the Settings tab; and then click on Identification.    Change the server name to something different than the default -- e.g. "NewUnRAID".    Be sure you click the "Apply" button.    Then go to the Main tab; Stop the array; and Reboot the server.

 

(3)  Be sure you can now access the server using the new name.    If not, reboot your PC and try it then.

 

(4)  Now boot your original v4.7 server and see if you can now access it okay.

 

Trurl/Gary

 

Thank you !!, yes, was named Tower, which I have now changed and both servers are running as I type. I did change the IP address to fixed on the new server which i assume would have been the same as 4.7? is that a problem? based on what you said I changed it back so it finds until the other issue was sorted out.

 

Can both servers operate on the same fixed IP?

 

I have discovered I that the power on switch is not operating presumably a cable issue, so I am connecting using IPMI.

 

Its not clear to me yet how I actually save anything to the new server, maybe I have to do a lot of reading. I don't really understand it if I am honest.

 

Please tell me how I upgrade 4.7 to the software the new server uses;

 

Is there any danger they the material currently stored will be lost?

 

How do I undertake this piece of work work?

 

Peter

 

Both servers can have a fixed IP, but they must have different IPs. Personally I just use DHCP and and have my router reserve IPs by MAC.

 

Upgrading to UnRAID v6

trurl

 

Thanks for your reply, i have established separate IP's for each server fixed, both running as I type. I have created top line shares.

 

I have some drives to put in which of course need to be pre-cleared  using XFS, I am assuming the there is some code somewhere for that?

 

I shall commence reading on the upgrade of 4.7 which you have kindly attached. I do have a few concerns about the current data as, at this time I am not able to put it anywhere. It would be a big confidence builder to here from someone who has successfully upgraded 4.7 to 6.xxxxx without issue.

 

Peter

 

Peter

I have some drives to put in which of course need to be pre-cleared  using XFS, I am assuming the there is some code somewhere for that?

Preclearing and XFS are not really related.  Preclearing drives thoroughly tests them, zeroes them, and prepares them for addition to an unRAID array.  XFS, ReiserFS, and BTRFS are file systems that can be added to data drives when they are added to the array.  I recommend using the Preclear plugin with the Fast Preclear script, after you have upgraded to v6.

 

I shall commence reading on the upgrade of 4.7 which you have kindly attached. I do have a few concerns about the current data as, at this time I am not able to put it anywhere. It would be a big confidence builder to here from someone who has successfully upgraded 4.7 to 6.xxxxx without issue.

I believe most unRAID users have upgraded now, with no risk to their data.  The upgrade should not affect the data drives, only the flash drive and the software that manages the unRAID array.  It's basically an upgrade of your flash drive.  One of the first steps is to make a complete backup of your flash drive, so if you wish later, you can restore it and run your v4.7 again, with the same data drives.

 

However, it's always wise to have a multiple backups of your more critical files.  While unRAID provides some fault tolerance to what is stored on it, files that are located only on your unRAID server are not backed up.  A file that is backed up is stored on multiple sites.  A copy on unRAID is only one copy, although unRAID does try to make it a safer copy.  That's an important distinction.

A few thoughts on your last couple posts (Rob has already noted some of this) ...

 

First, your issue was the naming conflict -- nothing to do with your IP's.    You can simply let both servers get their addresses via DHCP ... or, if you prefer, can set them both to static IP's => the important thing is they have to be different.    This will automatically be the case if you use DHCP, but if you assign them you need to be sure that (a) you use different IP's;  (b) they're in the same subnet as your router is assigning; and © that they're outside the range that the router will assign, so you don't have an IP conflict with some other device.    I'd just use DHCP.

 

As for storing stuff on the new server => you simply copy the data you want to the shares you create.    If you want to write directly to the disk shares, you have to enable them -- by default they are NOT enabled with v6.  Go to Settings - Global Share Settings - Enable Disk Shares to turn them on.

 

As Rob noted, pre-clearing is completely different than formatting the disks.    It just writes zeroes to the disks, and exercises them a good bit to confirm they're okay.    A couple of passes of pre-clear is an excellent test for a new disk.    The file system they'll be formatted in depends on what you've got set as the system default -- which in v6 defaults to XFS if you haven't changed it. [settings - Disk Settings - Default file system]

 

r.e. updating your old system => it's not too involved, but first I'd get the new server finished and be sure you're comfortable with the interface.

 

Is the new server going to have enough space to copy all of the data from the old server to it?    That would give you a complete backup of your data, which would mitigate any risk you might have in updating the old server.    As Rob noted, it's not difficult to do the update => the best approach is to completely reformat the flash drive in the old server to v6 and then assign the disks to it => but be CERTAIN that BEFORE you do that you have (a) saved your UnRAID key file from it; and (b) you know which disks (by serial number) were assigned to which slots in v4.7 [the important ones are the parity drive and cache drive (if you have one) ... the data drives can actually be assigned in any order].    As I noted, I'd get the new server working well first; and THEN do the upgrade => you can post here when you're ready to do that if you'd like confirmation of the exact steps you're getting ready to do.

 

 

I think maybe I did not explain myself very well 

 

The new server is running, I have set some shares up on it. Currently there is only the parity drive and one other drive in it, so its for the new server that I want to add the new drives to. Definitely not enough capacity at this time to copy over data from the old box.

 

I wanted to put V6 on the old server to give some data security, and myself a little breathing space to get the drive count on the new machine up.

 

Only issues I currently have is the power switch on the box is not operating I think a cable connection issue so I am powering up from IPMI.

 

Thank you for duly noted comments I will await the necessary guidance to update 4.7. After the process of recovery I am anxious not to make any mistakes.

 

Peter

I think I have all the relevant information to hand and would like to proceed, I dont have a flash drive as such but a Kingston reader SD card just a bit worried about formatting it.

 

Peter

 

I think I have all the relevant information to hand and would like to proceed, I dont have a flash drive as such but a Kingston reader SD card just a bit worried about formatting it.

 

Peter

 

Not all SD card readers work with UnRAID => be sure you get a unique GUID with it and try it with the free demo before buying a license.    Personally, I'd get a new flash drive ... these are VERY inexpensive these days (< $10 and even < $5 in many cases).

 

I think I have all the relevant information to hand and would like to proceed, I dont have a flash drive as such but a Kingston reader SD card just a bit worried about formatting it.

 

Peter

 

Not all SD card readers work with UnRAID => be sure you get a unique GUID with it and try it with the free demo before buying a license.    Personally, I'd get a new flash drive ... these are VERY inexpensive these days (< $10 and even < $5 in many cases).

Agree. Now that it is so easy to transfer your license to a new flash drive I don't see any good reason to bother with SD readers anymore.

I don't have a flash drive at this time and was rather hoping to be able to do this upgrade, the system works fine i dont have any problems with it. I am just not sure how its formatted.

There does not appear to be a facility to format, so delete, copy and paste?. I want the keep the SD card its worked flawlessly, in fact I have one in the new machine.

 

I need to know also what I type for fast preclearing I have two new drives to go in the new server, can both be done together?

 

Peter

They are both Kingston readers and both currently running UnRaid 4.7  & 6. respectively. is there any problem with overwriting the existing then. I just dont want to make any changes to something that already works ok.

 

Peter

I don't have a flash drive at this time and was rather hoping to be able to do this upgrade, the system works fine i dont have any problems with it. I am just not sure how its formatted.

Same as preparing a normal flash drive. See the Upgrading to V6 wiki. And search tips in my sig.