Best way to format - then move data?

[WannaTheater]

Hi -

On Friday I finished building my unRaid server, and I've been playing around with the 3-disk freebie (with intent to upgrade soon  ).  Here are my concerns:

 

1) The 3 drives I am using were unformatted...  unRaid build the parity drive (which was unformatted also).... I hope unRaid took care of formatting, because I didn't....  (... I kind of doubt it, because formatting of 3x500G drives would have taken much longer than it did.

 

Either way, I've been writing data to the 2 data drives all day with no apparent problem.

 

2) It is taking about 1.5 hours to move about 30G of data from my HTPC (3.4G P4 - 2G RAM/100Mbps ethernet) to the unRaid.  At this rate it will take me MONTHS to move the 3.0 TB that I need to move.  Without upgrading to gigabit, another method was mentioned:  use an external USB drive to transfer data (bypassing ethernet).

 

So:

 

a) After loggin in through telnet, how can I mount a USB drive in unRaid?

b) Which commands can be used to move data from mounted USB drive to unRaid discs, which will create the parity stuff, etc.  Standard "cp source destination"??

 

Thanks!

 

 

 

 

[foo_fighter]

Unraid formatted my 500gig drive unbelievably fast so it might have been a "quick" format.

 

Read http://lime-technology.com/forum/index.php?topic=239.0

for usb mounting instructions

 

However, I've only been getting 5MB/s doing this which is about the same as you're getting now. I don't know what the issue is. The usb disk is NTFS so it might be that.

 

I've been using

rsync -avu source_dir destination_dir

-avuz (compression) didn't help much since my files were already compressed.

 

You can also try turning off parity(change it to unassigned), copy over everything, and then re-generate parity. Copying automatically generates parity which is why it's so slow, it has to write,read,write everything.

 

Please update us if anything helps with the transfers.

 

Hi -

On Friday I finished building my unRaid server, and I've been playing around with the 3-disk freebie (with intent to upgrade soon  ).  Here are my concerns:

 

1) The 3 drives I am using were unformatted...  unRaid build the parity drive (which was unformatted also).... I hope unRaid took care of formatting, because I didn't....  (... I kind of doubt it, because formatting of 3x500G drives would have taken much longer than it did.

 

Either way, I've been writing data to the 2 data drives all day with no apparent problem.

 

2) It is taking about 1.5 hours to move about 30G of data from my HTPC (3.4G P4 - 2G RAM/100Mbps ethernet) to the unRaid.  At this rate it will take me MONTHS to move the 3.0 TB that I need to move.  Without upgrading to gigabit, another method was mentioned:  use an external USB drive to transfer data (bypassing ethernet).

 

So:

 

a) After loggin in through telnet, how can I mount a USB drive in unRaid?

b) Which commands can be used to move data from mounted USB drive to unRaid discs, which will create the parity stuff, etc.  Standard "cp source destination"??

 

Thanks!

 

 

 

 

[WannaTheater]

I have an update.... In an attempt to increase throughput, I purchased a gigabit switch.  The new switch did not appear to provide me with any substantial increase in moving data to unraid.  So at 30gig per 90 minutes, it will take about 24 hous of straight writing to fill a single 500G drive...  And I need to fill about 9 of them.

 

Is there something way off with my write performance?  Is there something I am overlooking?

 

 

 

[limetech]

Does your HTPC also have a GigE port?  If not, the GigE switch won't help.

 

I've seen as high as 9MB/s on a 100Mb/s network, though we haven't run any non-GigE testing lately - it's possible some GigE tuning has slowed down non-GigE performance, though not likely.  We should get a chance to test this later in the week.

 

Another possibility is one or more of your drives are not in DMA mode for some reason.  To check this you can use the 'hdparm' command, eg:

 

root@Server:~# hdparm -i /dev/hda

/dev/hda:

Model=ST3500841A, FwRev=3.AAD, SerialNo=3PM008AV
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
BuffType=unknown, BuffSize=8192kB, MaxMultSect=16, MultSect=16
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=268435455
IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes:  pio0 pio1 pio2 pio3 pio4
DMA modes:  mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
AdvancedPM=no WriteCache=enabled
Drive conforms to: device does not report version:

* signifies the current active mode

 

Look at the mode indicated by the asterisk (the above drive is in udma5 mode).

 

Do this for each of your hard drives.  You can look on the Devices page to get the device identifers to use for each disk.

[WannaTheater]

My HTPC does have an onboard GIG connection.

 

The PATA drives are in UDMA5.  I cannot get the command to work for the 4 SATA drives..

 

hdparm /dev/scsi/sdh2-2c0i0l0 returns:

 

/dev/scsi/sdh2-2c0i010:

HDIO_GET_IDENTITY failed: Inappropriate ioctl for device

 

Not sure if this matters, but all my SATA drives are Western Digital.  My 2 PATA drives are Seagate.