A couple days ago I yanked out the empty EARS from the array, rebuilt and as scheduled the 8/1 check rain.  Speed was up to about 85 Mb/s, not speedy but in the right direction.

 

Here is the plan going forward:

For now I'm only replacing the other EARS drive, will pick one soon (see question below).

My current warm spare is the Toshiba, but I don't have any in use.  Last time I bought a drive, I was seeing a lot of complaints of failures at 1 yr on the ST3000DM001, so bought the Toshiba.

 

I'm also getting a 2 port PCIe card, purely to use for the eSata ports (for preclears) so I can keep those MB ports free.

http://www.newegg.com/Product/Product.aspx?Item=N82E16816124064

Anyone ever use one in unRaid?

That will leave me 14 ports (6 on the MB / 8 on the SAS) so I wont need the old PCI card and away it goes. 

 

Around when 6.2 Stable is out, I'll get a 2nd parity drive and a 500GB SSD. 

My current SSD goes in this, so will get one more

http://www.newegg.com/Product/Product.aspx?Item=N82E16817998183&cm_re=SY-MRA25023-_-17-998-183-_-Product

Two of those and the 12 bays total of 14 available drive slots.

This also frees up the current position of the 500GB drive for the 2nd parity which happens to be above the current parity.

 

So my array is built of 2 and 3 TB drives now having 19.5 total capacity and 7TB free, so i'm not slim on space. 

When i replace the EARS next week or so with a 3TB, I'll have 20.5.

Over the next year I"ll replace the remaining slower 2 TB's which are getting up there in age anyway and that will bring me another 4TB added space 24.5

If you consider I could still add 2 more 3 TB drives in this case I'm at 30.5 and without ever moving off 3TB parity. 

 

Obviously I'm weighing the capacity options.  Conventional wisdom would tell me for the next two drives to be at least 4's if not 5's and make them parity drives so at some point I can use larger data drives.  However, the need seems pretty far off in the distance, If I consider I have capacity for 29.5TB on these parity drives and only 12.5 used to date I can go years, probably 5 at least.  Planning technology that far out seems unpredictable.  Hopefully we're on SSD's by then or who knows.

 

One question, if I were to use 4's or 5's, is it possible to format them as 3's for now, so parity checks to run longer?

 

It's not directly possible to format a 4 or 5TB drive as a 3TB drive;  but what you CAN do is use the Set Max function in HDAT2 [ http://www.hdat2.com/ ]  to set the size of one of the larger drives to 3TB.    This will make it "look" like a 3TB drive.

 

To later resize it would be a destructive process (i.e. you'd lose all the current data) => but if this was a parity drive all you'd have to do is a new parity sync, so the risk is fairly low [You'd be running "at risk" while you did that ... but as long as you do a parity check BEFORE you do the resize/resync to confirm all is well it's a very low risk.

 

 

Note there's an interesting performance benefit from doing that => with the drive set to only use 3TB (of 4 or 5) there won't be any data on the innermost cylinders, so the sustained data rate won't drop off nearly as much as it normally would.    Of course that won't really help with parity check speeds, since you have OTHER drives that are "real" 3TB drives and they'll be the bottleneck for parity check speeds.

It's not directly possible to format a 4 or 5TB drive as a 3TB drive;  but what you CAN do is use the Set Max function in HDAT2 [ http://www.hdat2.com/ ]  to set the size of one of the larger drives to 3TB.    This will make it "look" like a 3TB drive.

 

This creates an HPA on the drive that consumes the additional space.

 

I would recommend a full power down of the server before attempting this. Also you will want the drive hooked up to a motherboard port. Some controllers don't support this operation (although after HPA is in place, you can use with any controller.

It's not directly possible to format a 4 or 5TB drive as a 3TB drive;  but what you CAN do is use the Set Max function in HDAT2 [ http://www.hdat2.com/ ]  to set the size of one of the larger drives to 3TB.    This will make it "look" like a 3TB drive.

 

This creates an HPA on the drive that consumes the additional space.

 

I would recommend a full power down of the server before attempting this. Also you will want the drive hooked up to a motherboard port. Some controllers don't support this operation (although after HPA is in place, you can use with any controller.

 

So it looks like this is a dos program run from a bootable CD (or thumb drive).  So it seems you would have to shut down the server, remove the unraid thumb drive and boot off the DOS thumb drive to run this utility. 

 

Would you have to re-clear/preclear the drive after re-sizing?  I would presume unRaid would not recognize it as having been cleared yet and also would think I need/want to preclear the entire drive before re-sizing?

 

There are multiple ways to create an HPA. In Linux you can do with the hdparm command. See HERE for an example of how I used it to create and HPA on a 3T drive to make it appear as a 2.2T drive so that it would work with unRaid in the days before unRaid supported drives larger than 2T.

 

I have never used the tool that Gary references, but I think it works without needing to type a command on the command line.

 

As I mentioned before, do a full power down of the drive before attempting. Setting or removing an HPA only works once per drive power cycle. And use a motherboard sata slot. Some add-on controllers won't be able to create the HPA, although once created, any controller will recognize and respect the HPA.

So it looks like this is a dos program run from a bootable CD (or thumb drive).  So it seems you would have to shut down the server, remove the unraid thumb drive and boot off the DOS thumb drive to run this utility. 

 

Would you have to re-clear/preclear the drive after re-sizing?  I would presume unRaid would not recognize it as having been cleared yet and also would think I need/want to preclear the entire drive before re-sizing?

 

Yes, it's a DOS-based utility.  You can download either a bootable CD or the files to create a bootable USB flash drive.

 

You then simply boot to it and run the Set Max command as outlined on the web site.

 

Since your goal is to use the drive as parity, there's no need to re-run the pre-clear before adding it to the system.  You do, of course, want to run a couple pre-clear cycles to test the drive; but you can do this before you change the size if you want to ensure the entire drive is tested (a good idea).    But clearing the drive has no impact on how long it takes to do a parity sync, so there's no need to re-do that after you've changed the effective size.

 

There are multiple ways to create an HPA. In Linux you can do with the hdparm command. See HERE for an example of how I used it to create and HPA on a 3T drive to make it appear as a 2.2T drive so that it would work with unRaid in the days before unRaid supported drives larger than 2T.

 

I have never used the tool that Gary references, but I think it works without needing to type a command on the command line.

 

As I mentioned before, do a full power down of the drive before attempting. Setting or removing an HPA only works once per drive power cycle. And use a motherboard sata slot. Some add-on controllers won't be able to create the HPA, although once created, any controller will recognize and respect the HPA.

 

So use hdparm -NXXXX  where XX is the number of sectors on the disk that equals 3TB?    Then when I want to go back, re-use the same commend with the full sector count?  (lose everything, re-build parity). 

 

Seems pretty simple really.  no downloads or reboot on a dos disk. 

There are multiple ways to create an HPA. In Linux you can do with the hdparm command. See HERE for an example of how I used it to create and HPA on a 3T drive to make it appear as a 2.2T drive so that it would work with unRaid in the days before unRaid supported drives larger than 2T.

 

I have never used the tool that Gary references, but I think it works without needing to type a command on the command line.

 

As I mentioned before, do a full power down of the drive before attempting. Setting or removing an HPA only works once per drive power cycle. And use a motherboard sata slot. Some add-on controllers won't be able to create the HPA, although once created, any controller will recognize and respect the HPA.

 

So use hdparm -NXXXX  where XX is the number of sectors on the disk that equals 3TB?    Then when I want to go back, re-use the same commend with the full sector count?  (lose everything, re-build parity). 

 

Seems pretty simple really.  no downloads or reboot on a dos disk.

 

hdparm -N pXXXXXXXXXX

 

Some fear the command line.

HDParm is indeed the best choice if you're going to do this on the system you're using it on.

 

I always attach a disk to a spare system (as the only disk) when I'm doing anything alone these lines, as I don't want ANY chance that I inadvertently change the wrong disk      [Not that any of us would ever do anything like that  :) ]

 

For that, it seems easiest to just boot to HDAT2.

 

It's still a simple one-line command line process ... but with no other disks in the system it just seems a bit safer.  You could, of course, also boot to a CD-based Linux at that point and use HDParm (or, for that matter, an UnRAID USB flash drive)

 

 

 

hdparm -N pXXXXXXXXXX

 

Some fear the command line.

 

I never professed to be a Linux guy, but i crew up on DOS, an Apple II, a TRS-80, and the orignal IBM PC's.... no fear of "the command line" just in absence of a strong Linux background I'm liable to double/triple check around here to make sure I have it before leaping..

 

Thanks

 

hdparm -N pXXXXXXXXXX

 

Some fear the command line.

 

I never professed to be a Linux guy, but i crew up on DOS, an Apple II, a TRS-80, and the orignal IBM PC's.... no fear of "the command line" just in absence of a strong Linux background I'm liable to double/triple check around here to make sure I have it before leaping..

 

Thanks

 

and remember to do full power off after you apply a command above to make it permanent..

i have mine disks with small HPA cos 4TB Raid0 parity is a little bit smaller than 4TB drive. but this depends from controller to controller.

 

hdparm -N pXXXXXXXXXX

 

Some fear the command line.

 

I never professed to be a Linux guy, but i crew up on DOS, an Apple II, a TRS-80, and the orignal IBM PC's.... no fear of "the command line" just in absence of a strong Linux background I'm liable to double/triple check around here to make sure I have it before leaping..

 

Thanks

 

and remember to do full power off after you apply a command above to make it permanent..

i have mine disks with small HPA cos 4TB Raid0 parity is a little bit smaller than 4TB drive. but this depends from controller to controller.

 

Someone else on the forum ran into this and found a setting in the raid controller firmware to fix.

 

hdparm -N pXXXXXXXXXX

 

Some fear the command line.

 

I never professed to be a Linux guy, but i crew up on DOS, an Apple II, a TRS-80, and the orignal IBM PC's.... no fear of "the command line" just in absence of a strong Linux background I'm liable to double/triple check around here to make sure I have it before leaping..

 

Thanks

 

and remember to do full power off after you apply a command above to make it permanent..

i have mine disks with small HPA cos 4TB Raid0 parity is a little bit smaller than 4TB drive. but this depends from controller to controller.

 

Someone else on the forum ran into this and found a setting in the raid controller firmware to fix.

 

Depends on the controller. Areca has a default setting that causes the RAID size to be rounded down. But it can be disabled to provide full size use. But other RAID cards round down and do not have such an option.

I think I know the answer to this, but I can swap the 500 GB for a 480 SSD with no issues so long as I don't have over 480 stored on the drive, correct?  (it's actually only just under half full).

I think I know the answer to this, but I can swap the 500 GB for a 480 SSD with no issues so long as I don't have over 480 stored on the drive, correct?  (it's actually only just under half full).

 

From a parity standpoint, there are no files on a drive, just bits, 500GB of them in your case.  So no, you can't swap the SSD in.  You can add the SSD to the array, then copy the files over to it.

I think I know the answer to this, but I can swap the 500 GB for a 480 SSD with no issues so long as I don't have over 480 stored on the drive, correct?  (it's actually only just under half full).

 

From a parity standpoint, there are no files on a drive, just bits, 500GB of them in your case.  So no, you can't swap the SSD in.  You can add the SSD to the array, then copy the files over to it.

 

Makes sense, thanks, just saw a deal on a 480, but will stick with a 500.

I removed both EARS drive and one eliminated and one replaced with Toshiba.  I reconfigured the use of ports and right now 5 are on the MB, the others are on the SAS card, and i got rid of the old PCI-E card.

 

The parity check went up some to 87.4 avg.  I ran a disk speed check and all the drives are averaging 100 or greater. 

 

would the logical conclusion be the SAS card is the limiting factor?

In my experience a fully loaded SASLP limits parity check to around 75Mb/s, with 6 drives I get a little over 100Mb/s, try to have least amount of drives possible on the SASLP.

Also, I  would keep the 500Gb drive on the SASLP so it would speed up after that.

Forgot to add, make sure your spare is also connected do the SASLP because it won’t be used during parity checks.

 

If you don’t have it install the dynamix system stats and look at the graph after the check, on a non-limited system it should look like the one below.

 

Because you have different size disks there will be changes in speed at the 500GB and 2Tb marks, but the graph should still be curved, if there are big flat areas, especially at the beginning you are controller limited.

 

 

Forgot to add, make sure your spare is also connected do the SASLP because it won’t be used during parity checks.

 

If you don’t have it install the dynamix system stats and look at the graph after the check, on a non-limited system it should look like the one below.

 

Because you have different size disks there will be changes in speed at the 500GB and 2Tb marks, but the graph should still be curved, if there are big flat areas, especially at the beginning you are controller limited.

 

My parity check ran 3 days ago so the graph is a little squished down, but it bumped along at 740-780 MB/Sec for 6 hours, most of it around 750.  Then over the next 4 hours tapered down to 500.  So i'm reading that to say it was card limited for the first 6 hours. 

 

 

 

Another observation.  CPU usage runs from 93% (combined system and user) at the start of a parity check and tapers off down to 72 ish, more or less a steady taper for the total check.  Unlike the speed which flatlines as i described above.

So i'm reading that to say it was card limited for the first 6 hours.

 

Looks like you are but I would expect a little higher speed, the SASLP alone should give you around 600/640Mb/s, your CPU usage is also quite high.

 

There’s an easy way to test if you can move one or more drives from the SASLP to the board, if speed improves it’s the controller limiting you, if not it’s the board /CPU.

 

So i'm reading that to say it was card limited for the first 6 hours.

 

Looks like you are but I would expect a little higher speed, the SASLP alone should give you around 600/640Mb/s, your CPU usage is also quite high.

 

There’s an easy way to test if you can move one or more drives from the SASLP to the board, if speed improves it’s the controller limiting you, if not it’s the board /CPU.

 

I did this test a couple weeks ago when I first started this thread.  I still had the EARS drives on at that time, so will try again tomorrow night. 

 

Right now there is only one free port on the MB.  I'm only using 6 on the SAS card.  one port is empty for future and one has the warm spare attached.

 

I didn't think the CPU would be limiting a parity check, but I was surprised to see it running so high, and it i getting up there in age now.  I looked back, it's 5 1/2 years old, and wasn't exactly a speedy set up at the time.  I don't do much with add ons that would tax the CPU so didnt see the need when I first built it. 

So i'm reading that to say it was card limited for the first 6 hours.

 

Looks like you are but I would expect a little higher speed, the SASLP alone should give you around 600/640Mb/s, your CPU usage is also quite high.

 

There’s an easy way to test if you can move one or more drives from the SASLP to the board, if speed improves it’s the controller limiting you, if not it’s the board /CPU.

 

I moved one drive over from the SAS card to the MB (only one available port) and the speed check went from 87.4 to 90.2.  Only a little faster. I did notice when the 2 TB drives dropped off the speed was running well over 120 / 130.  Possibly a combination of less drives and faster drives only being left.

If the controller was the limit I believe you would see a bigger improvement, and with only five drives the SASLP limit is around 120-130Mb/s, so I think the board / cpu is your primary limitation.

 

If you haven’t yet run tunables tester, in some cases it can improve something like 20 or 30Mb/s.