ok here is my syslog right after i reset my motherboard settings.

so please dont feel irritated by the date/time

 

 

syslog.txt

bwm-ng bwm-ng v0.6 (refresh 1s); input: disk IO

Device Read Write Total

sda: 10224.41 KB/s 6047.24 KB/s 16271.65 KB/s

sdb: 5188.98 KB/s 6047.24 KB/s 11236.22 KB/s

total: 15413.39 KB/s 12094.49 KB/s 27507.87 KB/s

 

when i use copy within MC from one folder to another on the same disk

ok, i posted the syslog above. maybe you can figure something out.

 

my go file only exists with the unMenu boot string as well as with:

 

/boot/cache_dirs  -d  4  -m  3  -M  5  -w

 

maybe cache_dir is the reason?

when ever i shutdown the server i get serveral errors regarding

"chache_dir error /dev/hd[a..z] not found."

well i dont use IDE drives but maybe my script is broken or i misconfigured the (or any other) plugin wrong?

 

 

naaaahh this really drives me crazy.

 

 

can i somehow check if my discs performance is about normal?

why is my MC only at 9-10MB/s copy speed while writing from a disc to itself?

it could be a very very slow bus from the mobo, but i highly doubt it.

 

after looking into so many other issues for weeks now (that luckily could be solved thanks to you), i really would love to just start using unRAID 

 

 

today i'm back with more precise results.

i finally plugged in a 1000MBit switch.

 

(1. )

while my copy speed via 100MBit was of course only ~11MB/sec

now it is only at ~16-18MB/sec if I copy from windows to my unRAID server.

my switch reports 2 communicating gigabit network devices, so the connection itself should be good enough.

 

(2.)

now if I log into putty/telnet and try to copy from my one hdd to itself again, my speed isn't going higher than 16MB/sec, too.

 

 

well it is an increase by 80%, but I expected something from 30-60MB/sec at least to be honest.

 

Syslog

 

Jan 4 14:09:14 Tower kernel: e1000e: eth0 NIC Link is Down

Jan 4 14:09:14 Tower ifplugd(eth0)[1240]: Link beat lost.

Jan 4 14:09:22 Tower kernel: e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX

Jan 4 14:09:23 Tower ifplugd(eth0)[1240]: Link beat detected.

Jan 4 14:21:21 Tower shfs: shfs_rmdir: rmdir: /mnt/disk1/abc (39) Directory not empty

Jan 4 14:21:58 Tower in.telnetd[446]: connect from 192.168.178.27 (192.168.178.27)

Jan 4 14:22:00 Tower login[447]: ROOT LOGIN on `pts/0' from `.'

Jan 4 14:23:24 Tower shfs: shfs_rmdir: rmdir: /mnt/disk1/add (39) Directory not empty

Jan 4 14:29:11 Tower in.telnetd[2727]: connect from 192.168.178.27 (192.168.178.27)

Jan 4 14:29:13 Tower login[2728]: ROOT LOGIN on `pts/0' from `.'

 

 

ethtool eth0

root@Tower:~# ethtool eth0

Settings for eth0:

       Supported ports: [ TP ]

       Supported link modes:   10baseT/Half 10baseT/Full

                               100baseT/Half 100baseT/Full

                               1000baseT/Full

       Supports auto-negotiation: Yes

       Advertised link modes:  10baseT/Half 10baseT/Full

                               100baseT/Half 100baseT/Full

                               1000baseT/Full

       Advertised auto-negotiation: Yes

       Speed: 1000Mb/s

       Duplex: Full

       Port: Twisted Pair

       PHYAD: 1

       Transceiver: internal

       Auto-negotiation: on

       Supports Wake-on: pumbag

       Wake-on: g

       Current message level: 0x00000001 (1)

       Link detected: yes

 

 

ifconfig eth0

root@Tower:~# ifconfig eth0

eth0      Link encap:Ethernet  HWaddr 00:18:f3:c6:56:c3

         inet addr:192.168.178.24  Bcast:192.168.178.255  Mask:255.255.255.0

         UP BROADCAST NOTRAILERS RUNNING MULTICAST  MTU:1500  Metric:1

         RX packets:1448415225 errors:0 dropped:0 overruns:0 frame:0

         TX packets:655319876 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:1000

         RX bytes:2605991747 (2.4 GiB)  TX bytes:3717581153 (3.4 GiB)

         Memory:ff6e0000-ff700000

 

My Go- file

#!/bin/bash

# Start the Management Utility

/usr/local/sbin/emhttp &

cd /boot/packages && find . -name '*.auto_install' -type f -print | sort | xargs -n1 sh -c

/boot/unmenu/uu

/boot/cache_dirs  -d  4  -m  3  -M  5  -w

 

AHCI is not forced. The detection setting from bios is set to automatic. I guess i could force it, but I'm not experienced enough with this.

 

Copying from 1 hdd to another within my windows PC usually is around 63MB/sec or higher, depending of the hdds.

 

I really hope anyone is reading this and has some ideas or tests to suggest.

I so much need to get this issue done and finally start to do what i planned to do with the server.

Putting it out of sight, knowing it is there and working for a looong time.

 

Please help.

 

 

Log onto unRAID via telnet or on the system console.

 

Type

mc

to enter midnight-commander

 

Use it to copy from one disk to another.

All your disks will be found under /mnt

Copying using window's explorer has all the data going from the server to your windows PC and back, severely slowing you down.

Log onto unRAID via telnet or on the system console.

 

Type

mc

to enter midnight-commander

 

Use it to copy from one disk to another.

All your disks will be found under /mnt

Copying using window's explorer has all the data going from the server to your windows PC and back, severely slowing you down.

 

Well, was my post to long?

I already did that and reported it in the post above.

 

(1. )

while my copy speed via 100MBit was of course only ~11MB/sec

now it is only at ~16-18MB/sec if I copy from windows to my unRAID server.

my switch reports 2 communicating gigabit network devices, so the connection itself should be good enough.

 

(2.)

now if I log into putty/telnet and try to copy from my one hdd to itself again, my speed isn't going higher than 16MB/sec, too.

 

 

 

Short:

copying data from windows to unRaid -> ~17 MB/sec

copying data from unRaid to unRaid with mc -> ~16MB/sec.

 

 

 

Edit:

speed tests (using hdparm -tT) via user scripts from unMenu

 

/dev/sda:

Timing cached reads:   5658 MB in  2.00 seconds = 2832.75 MB/sec

Timing buffered disk reads:  304 MB in  3.01 seconds = 101.14 MB/sec

 

/dev/sdb:

Timing cached reads:   5886 MB in  2.00 seconds = 2947.22 MB/sec

Timing buffered disk reads:  304 MB in  3.02 seconds = 100.70 MB/sec

 

/dev/sdc:

Timing cached reads:   5372 MB in  2.00 seconds = 2689.02 MB/sec

Timing buffered disk reads:   40 MB in  3.16 seconds =  12.68 MB/sec

 

 

how could that be?

Is every data going through my USB Flash? .....

Nothing goes through your flash drive.

 

(sorry I missed your prior statistics)

 

The hdparm tests are only teating raw read speeds, and only on a single disk at a time.  hdparm does not test write speed.  Write speed is mostly limited by the rotational speeds of the disks and the bus throughput.

 

To see the maximum possible "write" speed you can type

 

dd if=/dev/urandom bs=1M of=/mnt/disk1/test_file.dat count=8000

 

It will write a file named test_file.dat that will be 8000 Megabytes in size (8Gig)

It will report the timing.   There is absolutely nothing in the network involved, but you will be limited by the slower of the rotational speeds of the parity disk and of disk1.   Both must spin the platters about 2 1/2 times per block of data written.

 

Once you've created the file, you can test the raw read speed with

dd if=/mnt/disk1/test_file.dat of=/dev/null bs=1M

 

To do the same test through a user-share, just type

dd if=/dev/urandom bs=1M of=/mnt/user/movies/test_file.dat count=8000

 

and

dd if=/mnt/user/movies/test_file.dat of=/dev/null bs=1M

(assuming "movies" is a valid user-share on your server)

 

Speed through a user-share will be slower.

 

when you were copying using "mc" from a disk to itself the time needed will be higher.  

Now, that one disk has to:

Position the read heads to read the existing block of data

Wait until the correct sector is under the read head

Read the existing block 1

Position the read heads to read the block of data to be written

Wait until the correct sector to be written is under the read head

Read the target block 1

(at the same time the parity disk is positioning the read head to the existing parity block and then

waiting until the correct sector is under the read head and then reading the sector)

When both the target sector AND the parity sector are read they are xor'ed to get the new parity calculation.

Then,  two "writes" are issued... once to the parity disk, and one to the "target" disk.

They each have to wait until the correct sector is under the write head.  (Typically, between a "read" and subsequent "write" the platter must spin at least once... but it can be more if other processes move the disk head, such as the process reading the original file on the source disk)

Now, the two sectors are written... but wait, the file copy is not complete, we need to move the read head back to the source file's location on the disk and wait for the correct sector to come around once more to read it.  position the read heads on both the parity disk and the target disk to the sector to be read and subsequently re-written.

 

I'm counting probably 5 revolutions of the platter on the disk being copied from and two or three on the one being copied to.  Moving a disk head is a very time consuming process, waiting for a cylinder to spin under the head equally so.  Even with buffering in the drives it takes time.

 

In the mean time we've written 16Gig (8 to parity, 8 to the data disk) and read 24 Gig (8 from the source, 8 from the target, and 8 from parity) all through your motherboard's bus.  That's 40 Gig of data.

 

Joe L.

thanks for the tips.

 

i will test all things you suggested and report my results.

 

in the meantime can you tell me what write -speed rates a healthy system should bring up?

thanks for the tips.

 

i will test all things you suggested and report my results.

 

in the meantime can you tell me what write -speed rates a healthy system should bring up?

I just corrected my prior post.

 

The copy of random data should be from /dev/urandom, not /dev/random.  (not the added leading "u" in "urandom")

 

Joe L.

/here are some representative timings from my newer server.

 

Apparantly, reading from /dev/urandom takes a LONG time.  Reading from /dev/zero is much faster.

 

In any case, I tried it with 4096 as the block size, and with 1Meg as the block size.  Not too much difference.

root@Tower2:~# dd if=/dev/urandom bs=1M of=/mnt/disk1/test_file.dat count=8000

8000+0 records in

8000+0 records out

8388608000 bytes (8.4 GB) copied, 2597.51 s, 3.2 MB/s

root@Tower2:~# dd if=/dev/urandom bs=4096 of=/mnt/disk1/test_file.dat count=2000000

2000000+0 records in

2000000+0 records out

8192000000 bytes (8.2 GB) copied, 2517.17 s, 3.3 MB/s

root@Tower2:~# dd if=/dev/zero bs=4096 of=/mnt/disk1/test_file.dat count=2000000

2000000+0 records in

2000000+0 records out

8192000000 bytes (8.2 GB) copied, 239.899 s, 34.1 MB/s

root@Tower2:~# dd if=/dev/zero bs=1M of=/mnt/disk1/test_file.dat count=8000

8000+0 records in

8000+0 records out

8388608000 bytes (8.4 GB) copied, 264.704 s, 31.7 MB/s

root@Tower2:~# dd if=/mnt/disk1/test_file.dat of=/dev/null bs=4096

2048000+0 records in

2048000+0 records out

8388608000 bytes (8.4 GB) copied, 89.7247 s, 93.5 MB/s

root@Tower2:~# dd if=/mnt/disk1/test_file.dat of=/dev/null bs=1M

8000+0 records in

8000+0 records out

8388608000 bytes (8.4 GB) copied, 90.0364 s, 93.2 MB/s

 

And here is my older server, wiriting to an old (and much slower) 400Gig IDE drive

root@Tower:/boot# dd if=/dev/zero bs=4096 of=/mnt/disk2/test_file.dat count=2000000

2000000+0 records in

2000000+0 records out

8192000000 bytes (8.2 GB) copied, 1275.85 s, 6.4 MB/s

root@Tower:/boot# dd if=/dev/zero bs=1M of=/mnt/disk2/test_file.dat count=8000

8000+0 records in

8000+0 records out

8388608000 bytes (8.4 GB) copied, 1009.16 s, 8.3 MB/s

root@Tower:/boot# dd if=/mnt/disk2/test_file.dat of=/dev/null bs=1M

8000+0 records in

8000+0 records out

8388608000 bytes (8.4 GB) copied, 794 s, 10.6 MB/s

root@Tower:/boot# dd if=/mnt/disk2/test_file.dat of=/dev/null bs=4096

2048000+0 records in

2048000+0 records out

8388608000 bytes (8.4 GB) copied, 757.329 s, 11.1 MB/s

not sure if its been solved or not, but I am experiencing slow copy speeds myself.

 

my drive is a 20ears with pins 7/8 jumpered.

 

using putty and the MC command I try the following.

 

move file from /mnt/disk1/Downloads/Complete/Movies/MoieName  /mnt/disk1/Movies/

 

I get about 10MB/s transfer speed.  but if I go from  say disk2 or a NTFS drive mounted using unMenu outside the array I get much faster write speeds.

 

I guess I need a cache drive  asap.

 

-=Jason=-

 

EDIT: using Putty at work I moved 2 large files (one 5gb and one 3gb) using MC from /mnt/disk1/Downloads/Complete/Movies  to /mnt/disk1/Movies and it was lightening fast, I did't even see a speed it just did it.

 

so im thinking my win7 laptop is some how to blame, not sure how or why if I am using putty and mc to control the server how can win 7 laptop be bottle necking my transfers.??

 

-=Jason=-

EDIT: using Putty at work I moved 2 large files (one 5gb and one 3gb) using MC from /mnt/disk1/Downloads/Complete/Movies  to /mnt/disk1/Movies and it was lightening fast, I did't even see a speed it just did it.

Moving a file on the SAME disk is just a matter of creating the directory entries that point to the file.  It will occur in a fraction of a second.

 

Moving a file from one physical disk to another requires you to read the source file (let's say at 80 MB/s) and reading the target data blocks to be written, and the parity blocks to be written, and then writing the data blocks and the parity blocks, all while waiting for the disk cylinders to rotate between the read, the write, and the subsequent read.  Let's say the overall rate is 30 MB/s.

 

So you have 8000 MB (8 gig) to read at 80MB/s = 100 seconds.

You have 8000 MB to write at 30MB/s  = 266 seconds

Total time = 366 seconds or roughly 6 minutes...    You're not likely to get too much better than this.

 

It will probably take twice that if done over the LAN.

Joe while at home on the Lan and Win 7 I am moving the file from the same disk.

 

/mnt/disk1/Downloads/Complete/Movies/MovieName to /mnt/disk1/Movies/

 

its the same disk, but at home on the win7 laptop using MC its painfully slow. at work not even on the lan but connected via putty its blazing fast.

 

using teracopy forget about its its at 1.5-3 MB/s  while mc is 10MB/s and at work its lightening fast

 

-=Jason=-

I've no idea what was going on, but I've transfered several files today using mc from the same disk and its been lightening fast.

 

does it make a different if you  "copy" vs "rename/ move" as I think I might have been doing a  F5 copy instead of F6 rename/move

 

-=Jason=-

Yes. The data does not get copied when you rename/move on the same filesystem, only the filename and directory path are changed.