July 24, 200916 yr I unmounted one of my drives and assigned it to cache duty to speed up my uploads. However the opposite has happened. I was getting 8MBps to over 10MBps writing directly to the Share (even the same disc now used as the cache). It started at 8MBps but quickly went down to less than 6MBps (5.41MBps as I type) . There are no other writes (just playing music as always) or parity check going on. What is the advantage of a cache if it's slower? Just to be clear I see it writing to the cache drive and the parity (and all other disks except the music Share disk) are spun down.
July 24, 200916 yr Author post your syslog... something does not sound right. My syslog is too large. it is 368KB. and won't upload. Can I just post part of it or should I break it up into 3 files?
July 24, 200916 yr Author If you use winzip or the like then you should be able to post it.. I forgot it's built into OS X.
July 25, 200916 yr I unmounted one of my drives and assigned it to cache duty to speed up my uploads. However the opposite has happened. I was getting 8MBps to over 10MBps writing directly to the Share (even the same disc now used as the cache). It started at 8MBps but quickly went down to less than 6MBps (5.41MBps as I type) . There are no other writes (just playing music as always) or parity check going on. What is the advantage of a cache if it's slower? Just to be clear I see it writing to the cache drive and the parity (and all other disks except the music Share disk) are spun down. Since file copies/moves don't show up in the syslog, I am not sure of the timeline here, so I will summarize what I can see, and that should help you figure out what may have been happening. * 04:08:37 Booted * 04:08:40 Array started * 04:08:41 Parity check starts (while it is running for the next 6 hours, EVERY thing else is going to be MUCH slower) * 04:08:41 Finishes 8 transactions replayed on Cache drive * 04:09:43 Finishes 271 transactions replayed on Disk 1 - all drive mounting is now complete too * 04:09:44 Begins creating the User Shares system, including identifying and reporting numerous duplicates * 04:37:25 Finishes finding and reporting duplicates - obviously very slow! * 04:39:41 Cache drive spins down - on 30 minute spin down timer, that indicates its last access was 04:09:41 * 06:17:43, 07:45:44, 08:51:44 Smaller drives spin down, parity check has finished with them * 09:49:48 Mover manually started * 09:50:22 Mover finished * 10:06:46 Cache drive spins down again, indicating there were file operations to it, completing at 09:36:46; these file operations would have been greatly impacted by the parity check * 10:21:06 Parity check completes, very good speed of 65571K/sec * 16:53:55 Last syslog entry - there were numerous file operations before this, and numerous drive spin downs, but no further mention of the Cache drive Copies to the Cache drive before 10:21:06 would have been very slow. Copies after that would have been at full speed, but there is no evidence here, that the Cache drive was used after the parity check finished. You will of course know when you were making those slow copies. The duplicates bother me, not because of their quantity, but because of their location. Apart from a few .DS_Store files (typical of Mac backups), ALL of the duplicates were reported to be on Disk 1(!), which by definition should be impossible. A duplicate means you have found something you had already found previously in the very same path, but since the duplicate identification process begins with Disk 1, there should be no duplicates found before Disk 2 and higher. I believe I have seen a problem like this once before, and the user finally found a particular problem or quirk as I recall, but I don't remember what it was. Perhaps that user will see this and speak up ... I wonder if it was symlinks to other disks they had created on Disk 1 ... Minor issue: at least 3 of the Seagate drives (sdb, sdi, sdj) appear to still have their SATA150 jumper installed. Please see the Improving unRAID Performance, Remove SATA150 Jumper section.
July 25, 200916 yr Author Thanks RobJ, Some of the duplicate problems may be related to the fact that at the time of boot up the (now) cache drive was just unassigned from drive1 and the the new drive 1 was installed/assigned/rebuilt. The unassigned former drive1 still had the files on it. I deleted them after I assigned it as the cache drive. Later today I will unassign and preclear it and re-post my syslog. I will also look for and remove the jumpers (as long as I don't have to uninstall them). What is the data write rate I should be getting using a cache drive?
July 26, 200916 yr Author Changed cache drives, precleard drive and rebooted Tower. First I copied straight to Share and got over 10MBps (confirmed no cache). Next I copied using cache and got below 7MBps (confirmed using cache drive). Isn't this at least backwards and shouldn't the cache offer a speed increase? What write speeds do others get using a cache? Attached is my syslog. Copying a few more ripped BD's over right now and it's hobbling along at the same <7MBps rate using the cache. Is there some enable command I'm missing to allow it to work at full speed?
July 27, 200916 yr I get about 30 MB/s transfer rates using a cache drive, and about 14 MB/s without one. All my drives are SATAII. Yours definitely sound slow.
July 27, 200916 yr I can't give you a good answer. I've examined carefully your syslog, and there is no evidence that I can see as to why your transfers are so slow. Your syslog is not telling me everything however. Usually, the network driver will indicate what connection speed it attained, but some do not, and yours is one that is keeping it a secret. Since all of the speeds you reported are consistent with only Fast Ethernet at 100mbps, not gigabit speeds, you should verify your network speed with the following console command: (for more info, see Console Commands for Networking) ethtool eth0 Ethernet at 100mbps has a practical max of about 10MB/s, gigabit at 1000mbps is required for faster speeds. Could you provide a little more detail about your testing, such as what commands or tools you were using, what paths on the unRAID server you were copying to, and how you determined for each whether it was using the Cache drive or not? Sorry for asking such basic questions, but I like to confirm I'm not working from wrong assumptions. Also, what is the 4 port SATA card you are using?
July 27, 200916 yr Author I can't give you a good answer. I've examined carefully your syslog, and there is no evidence that I can see as to why your transfers are so slow. Your syslog is not telling me everything however. Usually, the network driver will indicate what connection speed it attained, but some do not, and yours is one that is keeping it a secret. Since all of the speeds you reported are consistent with only Fast Ethernet at 100mbps, not gigabit speeds, you should verify your network speed with the following console command: (for more info, see Console Commands for Networking) ethtool eth0 Ethernet at 100mbps has a practical max of about 10MB/s, gigabit at 1000mbps is required for faster speeds. Could you provide a little more detail about your testing, such as what commands or tools you were using, what paths on the unRAID server you were copying to, and how you determined for each whether it was using the Cache drive or not? Sorry for asking such basic questions, but I like to confirm I'm not working from wrong assumptions. Also, what is the 4 port SATA card you are using? Thanks Rob, Just checked my MB on my PC (it's the oldest in the chain) and it a GB. My SATA card is the Rosewill RC-218 (though the cache was connected to the MB). I will do some test later but have a new and somewhat weird simi problem. Installed a new 1.5TB and uninstalled to removed the jumpers off 3 of my HD's (2-750GB, 1-1TB) and then reinstalled (damn them jumpers are super small). When I fired back up 1 of the 750GB drives must have had a bad connection and unRAID automatically put the new 1.5TB drive in it's place and started a rebuild (not what I wanted). I shut down the array and re checked my cables and started back up. It still keep the same configuration (rebuilding) so I Stopped it and looked in Devices. The 750Gb was shown as unassigned so I unassigned the 1.5TB and assigned the 750GB back where it was. The array would not allow this saying the 750GB was too small even though it is the disk from that spot with all the original files. I guess once a rebuilt starts (even though it was only about 1%) only a disk the same size or larger can go there. Does this sound correct?
July 28, 200916 yr Just checked my MB on my PC (it's the oldest in the chain) and it a GB. Yes, it is a Realtek RTL8111C chipset, that supports 10mbps, 100mbps, and 1000mbps. On power on, it negotiates the best speed the cable connection can manage, so the fact that it is a gigabit NIC does not mean you will always get gigabit speed. It will *try* to set that speed, but the speed also depends on the quality of the network cabling, and gigabit support on the other end of the cable. And even if it can set it for gigabit speed between this machine and your router or switch, to obtain gigabit speed all the way to the desktop that you were doing the transfers from, depends on gigabit support all the way to that desktop. The command I gave you will tell you the speed of this leg of the network connection. Your actual transfer speed will only be as fast as the slowest leg. Installed a new 1.5TB and uninstalled to removed the jumpers off 3 of my HD's (2-750GB, 1-1TB) ... When I fired back up 1 of the 750GB drives must have had a bad connection and unRAID automatically put the new 1.5TB drive in it's place and started a rebuild (not what I wanted). I shut down the array and re checked my cables and started back up. It still keep the same configuration (rebuilding) so I Stopped it and looked in Devices. The 750Gb was shown as unassigned so I unassigned the 1.5TB and assigned the 750GB back where it was. The array would not allow this saying the 750GB was too small even though it is the disk from that spot with all the original files. I guess once a rebuilt starts (even though it was only about 1%) only a disk the same size or larger can go there. Does this sound correct? I *thought* (but perhaps wrongly!) that in that situation you should see the Web Management page showing you that disk slot with both the new drive plus the old drive in italics, and require checking off a "I'm sure I want to do this" box, before it would start the array and begin rebuilding the drive. You should be able to run the Trust My Array procedure, to put the array back the way it was. Do make sure all drives are correctly assigned, as they were before, and follow the instructions very carefully. It is good that you have so recently completed a parity check! Once the other problems are resolved, you should obtain a SMART report for your Cache drive (sde), see the Troubleshooting page, Obtaining a SMART report section. I don't anticipate any problems showing up, but since your slowest transfer speeds involved this drive, we might as well check it out, to make sure it is not the source of the slowness.
July 28, 200916 yr Author ...The command I gave you will tell you the speed of this leg of the network connection. Your actual transfer speed will only be as fast as the slowest leg. ...You should be able to run the Trust My Array procedure, to put the array back the way it was. Do make sure all drives are correctly assigned, as they were before, and follow the instructions very carefully. It is good that you have so recently completed a parity check! Once the other problems are resolved, you should obtain a SMART report for your Cache drive (sde), see the Troubleshooting page, Obtaining a SMART report section. I don't anticipate any problems showing up, but since your slowest transfer speeds involved this drive, we might as well check it out, to make sure it is not the source of the slowness. I let it rebuild over night so no big deal. I'm now preclearing the 750GB to remount in the array. Oddly I was not thinking since I can't run ethtool eth0 until preclear is done (get used to multi operating systems and forget about single command line systems). I did copy a test file back and forth to my Mac Pro and got about the same <10MBps speed both ways. So there must be a problem with my network. My router is fairly new (a N wireless) so I don't think that could be the bottleneck. I can't think of any other thing that they have in common (other than the wires) that could be the problem. Also I shut down my PC and restarted (since you said it auto detected power up) with same results.
July 28, 200916 yr So there must be a problem with my network. My router is fairly new (a N wireless) so I don't think that could be the bottleneck. I can't think of any other thing that they have in common (other than the wires) that could be the problem. What exact make/model wireless N router? Are you doing the file copies when testing the speed over a wireless link, or over a wired LAN? or a combination of both? If copying over cables, What type cables? (Are they cat5, cat5e, or cat6? ) Or, were they unmarked? Did you make the cables? (attach the rj45 connectors on the ends yourself?) Joe L.
July 28, 200916 yr Author What exact make/model wireless N router? Are you doing the file copies when testing the speed over a wireless link, or over a wired LAN? or a combination of both? If copying over cables, What type cables? (Are they cat5, cat5e, or cat6? ) Or, were they unmarked? Did you make the cables? (attach the rj45 connectors on the ends yourself?) Joe L. It's a Linksys WRT300N Using wired. Wireless is for my MacBook Air Apple Airports (for music server) and iPhone. The house was prewired with cat5 (probably).
July 28, 200916 yr What exact make/model wireless N router? Are you doing the file copies when testing the speed over a wireless link, or over a wired LAN? or a combination of both? If copying over cables, What type cables? (Are they cat5, cat5e, or cat6? ) Or, were they unmarked? Did you make the cables? (attach the rj45 connectors on the ends yourself?) Joe L. It's a Linksys WRT300N Using wired. Wireless is for my MacBook Air Apple Airports (for music server) and iPhone. The house was prewired with cat5 (probably). OK... the plot thickens... Cat5 is NOT sufficient for Gigabit speed LAN use. You need Cat5e. or Cat6. Then, there are two different standards for wiring RJ45 LAN connectors to the cable. One standard is for telephone use, the other for LAN use. They pair up the wires differently. (There are also two more minor variations of the LAN standard, but they just differ in the color of the pair used, not the actual pin-to-pin wiring.) I wired my own home with cat5e. The crimper I purchased has a chart on its cover showing the color-code to use... for TELEPHONE USE. If I were to follow it, my LAN would get terrible performance if it worked at all. (it all depends on how well the network cards can work with the degraded signal) The crimper made no mention of the other LAN wiring standard... obviously, it was marketed towards a telephone installer. If the person who wired your home used the wrong standard, or cat5, or cat3 cables, or did not keep the pairs of wires tightly twisted all the way into the connectors, anything is possible. There is an easy way to quickly test... Purchase a pre-made cat5e 50, 75, or 100 foot cable and run it in place of the existing cabling from the router to the PC. Run it down the halls, up the stairs, etc. Make it safe so you don't trip on it, explain to the family it is only temporary while you test. If it works, great. You can replace the temp connection as you re-wire your house. If the speed still stays as slow, we'll need to look at other possibilities. Joe L.
July 29, 200916 yr Author Just ran ethtool and here is what I got. Don't know how to capture console screen (or even if you can) so I copied from one of Joe's posts and changed what was different (in italics). I see I'm only getting 100Mbs. Can I manually change it to 1Gbps using the syntax ethtool -s eth0 speed 1000 full duplex autoneg off and if so what will happen? Settings for eth0: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised auto-negotiation: Yes Speed: 100Mb/s Duplex: Full Port: MII PHYAD: 0 Transceiver: internal Auto-negotiation: on Supports Wake-on: umbg Wake-on: g Current message level: 0x00000033 (51) Link detected: yes
July 29, 200916 yr Unless I'm mistaken, your Linksys WRT300N router is only a 100Mbit switch. There is no way to get it to work at 1Gbps speed. A quote from the WRT300N manual There's also a built-in 4-port full-duplex 10/100 Switch to connect your wired-Ethernet devices together. If you attempt to change the speed of your ethernet port to 1Gbps, it might not connect at all. (I've never tried, but since your router is not able to work at that speed, odds of a good result are not very high) Joe L.
July 29, 200916 yr Author Unless I'm mistaken, your Linksys WRT300N router is only a 100Mbit switch. There is no way to get it to work at 1Gbps speed. A quote from the WRT300N manual There's also a built-in 4-port full-duplex 10/100 Switch to connect your wired-Ethernet devices together. If you attempt to change the speed of your ethernet port to 1Gbps, it might not connect at all. (I've never tried, but since your router is not able to work at that speed, odds of a good result are not very high) Joe L. You know I pulled the box down and looked, loaded the PDF manual and looked at a couple of reviews and didn't see the speed of the Ethernet ports listed. So even though it's a fairly new (with N wireless) it could be just 100Mbs? While I don't want to buy a new router there is no way I can rewire my house. How can I get a definitive answer on the port speed question? EDIT: Looks like you are correct Joe (as if I would ever doubt it ). Here is the specs from Newegg: Ports 1 x 10/100M WAN; 4 x 10/100M LAN
July 29, 200916 yr You know I pulled the box down and looked, loaded the PDF manual and looked at a couple of reviews and didn't see the speed of the Ethernet ports listed. So even though it's a fairly new (with N wireless) it could be just 100Mbs? While I don't want to buy a new router there is no way I can rewire my house. How can I get a definitive answer on the port speed question? A lot of wireless routers are only 100Mbs. The cheap wireless N routers are not 1 Gbps speeds. Plus you also have the uncertainty of not knowing what is in your walls. If you buy a new router that has gigabit speed you might find that the wire in your walls is not up to snuff and the new router is pointless.
July 29, 200916 yr Unless I'm mistaken, your Linksys WRT300N router is only a 100Mbit switch. There is no way to get it to work at 1Gbps speed. A quote from the WRT300N manual There's also a built-in 4-port full-duplex 10/100 Switch to connect your wired-Ethernet devices together. If you attempt to change the speed of your ethernet port to 1Gbps, it might not connect at all. (I've never tried, but since your router is not able to work at that speed, odds of a good result are not very high) Joe L. You know I pulled the box down and looked, loaded the PDF manual and looked at a couple of reviews and didn't see the speed of the Ethernet ports listed. So even though it's a fairly new (with N wireless) it could be just 100Mbs? While I don't want to buy a new router there is no way I can rewire my house. How can I get a definitive answer on the port speed question? EDIT: Looks like you are correct Joe (as if I would ever doubt it ). Here is the specs from Newegg: Ports 1 x 10/100M WAN; 4 x 10/100M LAN You can purchase an inexpensive 1GBps switch (not wireless, not a router, just a switch) I have exactly that in my home. My wireless router connects to the cable-modem at 100Mbps, and to the 1Gbps switch at 100Mbps. I have several older devices plugged into the other ports on the router. They are all older 100MBps devices. The 1Gbps switch I purchased was on sale one day for about $30 at comp-usa. It is an 8 port switch. I have most of the other devices (and all the 1Gbps devices ) attached to it. It has the unRAID server connected to it. You can locate the 1Gbps switch on the same shelf as your current router. If it is able to get 1Gbps on your existing cabling it will. (If not, I guess you can always return it) I use this "netgear" switch, currently on sale for $39 after rebate: http://www.newegg.com/Product/Product.aspx?Item=N82E16833122141 But you might not need 8 ports, if not this might work with 5 ports at $24. http://www.newegg.com/Product/Product.aspx?Item=N82E16833122140 No idea if they are the best, or worst switches, but it was on sale for me when I purchased it, and it works well for me. Switches like these are pretty inexpensive, and easy enough to find locally if you do not want to order over the web. Joe L.
July 29, 200916 yr Author ...You can purchase an inexpensive 1GBps switch (not wireless, not a router, just a switch) I have exactly that in my home. My wireless router connects to the cable-modem at 100Mbps, and to the 1Gbps switch at 100Mbps. I have several older devices plugged into the other ports on the router. They are all older 100MBps devices.... 5 ports at $24. http://www.newegg.com/Product/Product.aspx?Item=N82E16833122140 No idea if they are the best, or worst switches, but it was on sale for me when I purchased it, and it works well for me. Switches like these are pretty inexpensive, and easy enough to find locally if you do not want to order over the web. Joe L. Thanks Joe, Just ordered the $25. Will report back Monday or so on results.
July 29, 200916 yr ...You can purchase an inexpensive 1GBps switch (not wireless, not a router, just a switch) I have exactly that in my home. My wireless router connects to the cable-modem at 100Mbps, and to the 1Gbps switch at 100Mbps. I have several older devices plugged into the other ports on the router. They are all older 100MBps devices.... 5 ports at $24. http://www.newegg.com/Product/Product.aspx?Item=N82E16833122140 No idea if they are the best, or worst switches, but it was on sale for me when I purchased it, and it works well for me. Switches like these are pretty inexpensive, and easy enough to find locally if you do not want to order over the web. Joe L. Thanks Joe, Just ordered the $25. Will report back Monday or so on results. Don't forget to print your rebate form from here: http://images10.newegg.com/UploadFilesForNewegg/rebate/SH/Netgear13MIRsJul1Sep3009jh28.pdf
July 29, 200916 yr Author Don't forget to print your rebate form from here: http://images10.newegg.com/UploadFilesForNewegg/rebate/SH/Netgear13MIRsJul1Sep3009jh28.pdf I actually bought a Rosewill (house brand) so don't have to deal with the rebate.
July 29, 200916 yr Meh I don't trust Rosewill... although the PCI-E x1 SATA cards I got from them have worked fine so far. Then again I only paid $5 a piece for them haha. I picked up a Trendnet 8-Port Gigabit switch from Buy.com. It's another questionable company but it only cost me $10 in the end and I must say... the switch feels like it's very decent in terms of quality. Metal exterior and much heavier than I expected. The weight is a good sign to better quality components inside. So far the switch has worked very well for me and I don't regret the purchase whatsoever. My $10 price was a fluke... but it regularly goes on sale for $20-30. I do still recommend a more reputable company... but so far I'm not regretting this switch at all.
Archived
This topic is now archived and is closed to further replies.