I'm thinking I could use the preclear_disk.sh script with the -V switch to skip the pre-read and clear and go straight to the post-read verify. I could then run screen and do this for all the drives? I'd be able to view the initial MB/s read speed and see if it decreases when starting up more drives.

 

I know it's dangerous running preclear on disks with data, but as long as I use the -V to only post-read I'd be okay, right? It wouldn't change any bits on drives, only read them?

 

Is there a more elegant and proper way to stress the hard drive bandwidth to make sure the onboard and pcie cards are functioning properly? I've heard if IOZone, has anyone had success with that on an unRAID system without damaging data?

A parity check reports the total time in the syslog upon completion.

A parity check reports the total time in the syslog upon completion.

I'm not looking for the total time it takes. I want to know the total hard drive bus bandwidth is use while all hard drives are being read. Ya know, see if there's a bottleneck in my configuration. Make sure my PCIe card is working to spec. Make sure my onboard SATA ports are functioning properly.

And during a parity check all drives are being read at maximum throughput. Isn't this exactly the type of test you're trying to do?  At the end of the parity check the time elapsed will be in the syslog. Divide parity drive size by time and you have your answer. Better yet, the WebGUI will display the average MB/s for the completed parity check so you don't have to do the math.

 

Assuming all disks of the same size, if the average is lower than the average speed of your slowest drive, then you have a bottleneck somewhere. Even with disks of multiple sizes you should still easily be able to determine if you have a bottleneck.

 

You could also use some form of hdparm script. I remember seeing one on the forums here.

And during a parity check all drives are being read at maximum throughput. Isn't this exactly the type of test you're trying to do?

Well, again, I don't want to know the total maximum throughput. I have 5 drives using onboard SATA, 3 drives on one PCIe SATA card, and one drive on another PCIe SATA card (experimenting at the moment with this).

 

If my onboard SATA bus is the bottleneck, I would never know it. Same if my PCIe card with 3 drives happens to be in a bad and/or PCIe 1.x slot I'd never know.

 

I suppose I'll just have to run a preclear read only on all my drives to find out what I'm looking for. I'd be able to check each drive one by one to make sure they're running around 130MB/s. If I find one or more lower then I'd be able to track it down and reconfigure my setup for optimum performance. Either do this or I suppose I'll have to look into IOZone as I mentioned before.

Maybe this will give you what you want, although the drives are tested individually...

http://lime-technology.com/forum/index.php?topic=31073.0

If my onboard SATA bus is the bottleneck, I would never know it. Same if my PCIe card with 3 drives happens to be in a bad and/or PCIe 1.x slot I'd never know.

 

Testing each drive individually is certainly good for testing the throughput of the drive itself (i.e. is the drive itself underperforming) or testing the individual port it is plugged into.  But if you are looking for BOTTLENECKS, you need to be testing all drives at once, or at least all drives connected to each individual controller at once. 

 

Here's an example.  Let's assume you had 4 of the Seagate 1TB/platter drives connected to a 4-port 1x lane PCIe card in a v1.x slot.  A v1.x slot is is capable of 250MB/s per lane.  If it were a 1 lane card then its maximum throughput is 250MB/s.  If you ran a throughput test on each individual drive on that controller separately, they would all pass with flying colors as the drive is capable of about 180MB/s and the card is capable of 250MB/s.  But if you ran a throughput test on all 4 drives simultaneously you would see there is a problem as the 4 combined drives are capable of roughly 720MB/s throughput, yet your card can only handle 250MB/s.  You would therefore see each drive reporting roughly 62MB/s throughput and know you had a bottleneck somewhere.

Testing each drive individually is certainly good for testing the throughput of the drive itself (i.e. is the drive itself underperforming) or testing the individual port it is plugged into.  But if you are looking for BOTTLENECKS, you need to be testing all drives at once, or at least all drives connected to each individual controller at once. 

 

Here's an example.  Let's assume you had 4 of the Seagate 1TB/platter drives connected to a 4-port 1x lane PCIe card in a v1.x slot.  A v1.x slot is is capable of 250MB/s per lane.  If it were a 1 lane card then its maximum throughput is 250MB/s.  If you ran a throughput test on each individual drive on that controller separately, they would all pass with flying colors as the drive is capable of about 180MB/s and the card is capable of 250MB/s.  But if you ran a throughput test on all 4 drives simultaneously you would see there is a problem as the 4 combined drives are capable of roughly 720MB/s throughput, yet your card can only handle 250MB/s.  You would therefore see each drive reporting roughly 62MB/s throughput and know you had a bottleneck somewhere.

Precisely what I want to do. But I'm looking for a way other than using screen for multiple threads and running preclear_disk.sh read-only in each screen.

 

I kind of did this before when I was originally clearing 4 drives at once. I was able to see the bandwidth of each drive to make sure they each got their max speeds. I'm just a bit uneasy running preclear_disk.sh on drives that have data.

That diskspeed.sh script uses hdparm. So I tried an 'hdparm -t /dev/sdb' and I was able to do a disk read test. But it was too short. What I'd need is a script to keep repeating the 'hdparm -t' command. Then I could get really crazy and try the -T switch to test each drive using its onboard cache only. With one drive's cache I got a 4535.43 MB/sec test.

That diskspeed.sh script uses hdparm. So I tried an 'hdparm -t /dev/sdb' and I was able to do a disk read test. But it was too short. What I'd need is a script to keep repeating the 'hdparm -t' command. Then I could get really crazy and try the -T switch to test each drive using its onboard cache only. With one drive's cache I got a 4535.43 MB/sec test.

Run the diskspeed.sh script with the -i parameter set to a high number - it'll run the hdparm -t in a loop x number of times over the same spot on the drive - or you could use the -s parameter set to a high number and it'll test more areas on the drive - or both.