Vorlagen Posted July 23, 2008 Share Posted July 23, 2008 Hi all, Recently set up my unRAID server, things are running now and I am happy with its storage ability. While this thread was initially started concerning torrent ability, I have found the issue to be much more systemic, so I would like to put the torrent-specific chatter on the back burner. When watching normal video (no other unRAID activity going on) off of the server I get frequent hiccups, momentary freezes, etc. While watching HD, these hiccups occur much more frequently and last longer. I spent much time reading the forums, and found one individual who had a similar case. For him, bad hardware was corrupting the write on the HDs. This is not the case for me, as I can put a video file on unRAID, pull it off again, and have no problems with the video once it's back on my internal HDs. In addition to this, if I'm currently making any transfers with unRAID, either writing or reading, and attempt to watch video from the server, it hangs indefinitely and will not start until the transfer is complete. I am using a Netgear Gigabit switch, CAT6 cables, etc. I have a ABIT AB9 PRO board with latest BIOS and latest version of unRAID Pro. Are there any tests I can do to see what is the problem? Anyone have any ideas? I have also had trouble with torrenting and unRAID using Mac OS 10.5 Leopard and Vuze. While the torrent client recognizes the unRAID shares and disks, it hiccups if I have any other activity going on. For example: I am transferring a large number of files onto the unRAID server over gigabit LAN (several hours transfer). Starting a torrent and telling the data to save to unRAID keeps all of the torrents in the "Allocating Space" state until all of the other transfers are complete. Another example: If the torrent is currently downloading to unRAID, and I start transferring other files, the torrent errors out and I have to re-check it after the transfers finish. Please find attached my syslog. Thank you. Link to comment
bubbaQ Posted July 23, 2008 Share Posted July 23, 2008 What torrent software are you using and on what platform? Link to comment
Vorlagen Posted July 23, 2008 Author Share Posted July 23, 2008 This is using the newest version of Azureus (Vuze) and OS X Leopard 10.5.4. unRAID is 4.3.3 Link to comment
Vorlagen Posted July 27, 2008 Author Share Posted July 27, 2008 Still receiving this problem--cannot watch videos while doing any sort of transfer, etc. Any way to make sure my network/HD performance is up to snuff or if there is some sort of issue? Link to comment
NAS Posted July 27, 2008 Share Posted July 27, 2008 the nature of the torrent spec means that it can be grabbing literally thousands of different parts of the file to dish out to random people at one time. Its a disk thrasher. In the vuze settings look at limiting the number of peers you actively connect to and use common sense on various other settings Link to comment
Vorlagen Posted July 27, 2008 Author Share Posted July 27, 2008 Thanks for the input. I'm an experienced torrenter, and have carefully tuned my settings over the years. Videos have small stutters and see artifacts when playing back with it being the only activity from unRAID as well. Link to comment
WeeboTech Posted July 28, 2008 Share Posted July 28, 2008 When I do large transfers, I do it with rsync at the command line. My system can do a sustained 10MB without interference or pauses flushing the cache. Then again I have 8GB of ram. So anything that causes cache buildup then flushing is going to affect performance. Although I believe it should not, it still does. By using rsync --bwlimit=10240 source destination I can keep everything flowing nicely without any issues. This may need to be tuned per your system, lan speed and memory. In my environment I am running rtorrent directly on unraid and by using rsync with bandwidth limiting, I've never run into any issues. Link to comment
NAS Posted July 28, 2008 Share Posted July 28, 2008 Is the problem the same on files with small number of pieces/blocks and large numbers of pieces/blocks. Pop an example of the .torrent contents so we can put this in context Link to comment
Vorlagen Posted August 2, 2008 Author Share Posted August 2, 2008 NLS--What exactly happens to you as well? NAS, WeeboTech, and others--While this thread was initially started concerning torrent ability, I have found the issue to be much more systemic, so I would like to put the torrent-specific chatter on the back burner. When watching normal video (no other unRAID activity going on) off of the server I get frequent hiccups, momentary freezes, etc. While watching HD, these hiccups occur much more frequently and last longer. I spent much time reading the forums, and found one individual who had a similar case. For him, bad hardware was corrupting the write on the HDs. This is not the case for me, as I can put a video file on unRAID, pull it off again, and have no problems with the video once it's back on my internal HDs. Are there any tests I can do to see what is the problem? Anyone have any ideas? Link to comment
RobJ Posted August 2, 2008 Share Posted August 2, 2008 Vorlagen, almost by accident, I was reviewing the previous posts in this thread, to refresh my memory and see if I had missed something, and noticed you had a syslog attached above, but with zero downloads. I did not remember seeing one before, so downloaded it and discovered it was dated Today! Please don't edit old posts, as there is no notification to others of edits, and therefore almost zero chance of anyone ever seeing the changes. I noticed in your syslog that it appears that ALL of your Seagates still have their jumpers installed, limiting them to SATA I speeds and feature sets. Normally, I comment on this when I see it in a syslog, so if I missed it in your syslogs before, I apologize. Removing the jumpers won't dramatically increase speeds, but should provide a small improvement, should be detectable with your system I think. I recommend checking the Improving unRAID Performance wiki page, for this and other suggestions. Also you can test raw read speed of your drives before and after any changes by the command found on the Check Harddrive Speed wiki page. Your system performance seems very good, but your gigabit network driver (r8169) is not reporting the network connection speed, unlike other gigabit drivers. I suggest you go to a console or Telnet prompt and type ethtool eth0, which will display the network connection speed in the middle of its report. If it is not Speed: 1000Mb/s, then that is the main reason for your performance problems. Link to comment
Vorlagen Posted August 2, 2008 Author Share Posted August 2, 2008 Thanks for the quick reply, RobJ. You are right, I did edit my original post. As I figured out more of the issues I was updating the original post so people would be caught up--didn't think that it would keep people from checking, but you're right. Thanks for pointing it out. Thanks, I'll go through and remove all of my jumpers. In regards to my network: I have a gigabit switch that is connected to my computers and my unRAID server. This in turn is connected to a Linksys WRT54G Router (100/10, not gigabit). Can this Linksys bottleneck my system even if the 2 computers talking to each other are on the same gigabit switch? Link to comment
RobJ Posted August 2, 2008 Share Posted August 2, 2008 The Linksys should not be a problem, if it is not in the critical path. Gigabit speed is negotiated, and sometimes you may not be aware of a defective cable or connector or some point in the link with the playback machine, that is not running at full speed. That's why I suggest you try the ethtool eth0 command to verify you are getting the speed you know you should be getting. You should also check the properties of the network connection on your playback machine, to make sure it too has a true gigabit connection. Link to comment
Vorlagen Posted August 2, 2008 Author Share Posted August 2, 2008 Both the computer and the unRAID server say 1000. Link to comment
RobJ Posted August 2, 2008 Share Posted August 2, 2008 Good. What speed are you seeing when you transfer a file from the unRAID server to your playback machine? If you copy a one gigabit sized file, it should take about 40 seconds at 25MB/s, or 30 seconds at about 34MB/s. Link to comment
Vorlagen Posted August 2, 2008 Author Share Posted August 2, 2008 Thanks, just for clarification: 1 gigabit = 128MB, correct? Link to comment
RobJ Posted August 2, 2008 Share Posted August 2, 2008 Sorry, I should have said 'one gigabyte sized file', as in 1000MB or 1024MB. Link to comment
Vorlagen Posted August 2, 2008 Author Share Posted August 2, 2008 Alright... I took off my jumpers and added the line to increase the read-ahead buffer (though I know this won't change write speeds). I uploaded a 983MB file and it took 3 minutes 6 seconds: (983 megabytes) / (186 seconds) = 5.28494624 MBps. This doesn't look too good. I'm still have the same issues with watching SD/HD videos (ie it doesn't seem read has improved drastically). Please find attached my new syslog. Link to comment
RobJ Posted August 2, 2008 Share Posted August 2, 2008 And I can confirm your syslog now indicates 7 drives changed from 1.5 to 3.0 Gbps speed, meaning they now have their full SATA II capabilities. You can't test anything right now, because you have a full parity sync running, which works your server about as hard as it can. When you removed Disk 15, and set everything to New, you started a parity sync to build a new parity drive. You will have to wait for that to finish, to get any kind of performance from your server. Link to comment
Joe L. Posted August 2, 2008 Share Posted August 2, 2008 It looks like you are in the middle of a parity sync. Let it finish before doing any performance tests. Also, you might want to disable one of your Ethernet controllers. (apparently there are two on your motherboard) unRAID only supports one, and I'm not sure what the other will do for performance. Link to comment
Vorlagen Posted August 2, 2008 Author Share Posted August 2, 2008 Thanks to you, RobJ and Joe L. both. I'll let it lie overnight until the parity check is done and then test again in the morning. I appreciate your help thus far. Link to comment
Joe L. Posted August 2, 2008 Share Posted August 2, 2008 One or two more questions: What OS is running on the PC you are uploading to? Did you make the cables used to connect the PCs on your lan to the switch/unRAID server ? Can you type the following two commands and capture the output for us? ethtool eth0 ifconfig eth0 Joe L. Follows is the output from my server. root@Tower:~# ethtool eth0 Settings for eth0: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised auto-negotiation: Yes Speed: 1000Mb/s <----- looking for 1000Mb/s here Duplex: Full Port: Twisted Pair PHYAD: 1 Transceiver: internal Auto-negotiation: on Supports Wake-on: umbg Wake-on: g Current message level: 0x00000007 (7) Link detected: yes root@Tower:~# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:11:11:75:FB:7E inet addr:192.168.2.100 Bcast:192.168.2.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:68094993 errors:0 dropped:0 overruns:0 frame:0 <-- looking for zero errors, zero dropped packets here TX packets:77754921 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2856146567 (2.6 GiB) TX bytes:2078553873 (1.9 GiB) Link to comment
Vorlagen Posted August 2, 2008 Author Share Posted August 2, 2008 OS X 10.5.4 Leopard--new 3.06 GHz Core2 iMac. The cables are newly purchased Cat6 cables from Radioshack (I know, but only place close by for the quick trip). Same as you except it says PHYAD:0 instead or your PHYAD:1, and my current message level differes as well. Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Supports auto-neotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised auto-negotiation: Yes Speed: 1000Mb/s Duplex: Full Port: Twisted Pair PHYAD: 0 Transceiver: internal Auto-negotiation: on Supports Wake-on: umbg Wake-on: g Current message level: 0x00000033 (51) Link detected: yes No errors, no dropped packets: eth0 Link encap: Ethernet HWaddr 00:50:8D:9D:3B:FA inet addr:192.168.1.102 Bcast:192.138.1.255 Mask:255.255.255.0 UP BROADCAST NOTRAILERS RUNNING MULTICAST MTU:1500 Metric:1 RX packets:10422291 errors:0 dropped:0 overruns:0 frame:0 TX packets:5816783 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1693091105 (1.5 GiB) TX bytes:1416078178 (1.3 GiB) Interrupt:17 base address:0x8000 Link to comment
Vorlagen Posted August 2, 2008 Author Share Posted August 2, 2008 Alright--that 5MB/sec I posted about earlier were write speeds (I used the term uploading). I see that you want my read speeds though. In 2 separate tests: (985 megabytes) / (20.6 seconds) = 47.815534 MBps (1.01 gigabytes) / (25.4 seconds) = 40.7181102 MBps However, I still have highly noticeable hiccups playing both standard def and HD. This is even more curious when I do a read test from my USB 2.0 external: (940.7 megabytes) / (25.6 seconds) = 36.7460938 MBps So I'm reading from unRAID faster than I do from the USB 2.0 external drive. However, the exact same files stored on the external playback flawlessly (HD/SD). Keep in mind I've already ruled out file corruption since I can grab the video file back to my local machine and watch it without any problems. When watching the transfer bar go, it seems that it moves very fast for a small period of time and then will stop or stutter--I think this may be causing the problems. I have seen the same issues when writing large numbers of files to the unRAID server--periods of high volume movements followed by what seem to be just a stall. Any ideas? The read-ahead buffer is definitely correctly on 2048 for all the drives, and as you said my drives are in SATAII mode. Please find attached my newest syslog. Link to comment
Joe L. Posted August 2, 2008 Share Posted August 2, 2008 What PC operating system are you using to play the videos? ( I asked earlier, but you did not answer the question) Matching up the "packet sizes" on the network for best performance is the next thing to think about. But first, we need to know a bit more. As an example, if the PC is asking for 16000 bytes, but the server is supplying more, then the excess is probably being thrown away and then re-requested. What gigabyte switch are you using? (It might be the cause of the delays) Some are able to keep up with the speed, others might not, and the advertised speed is simply peak speed and wishful marketing. Have you tried a direct connection from the PC to the unRAID server with a "crossover" cable? (eliminating the LAN entirely and everything else on it) What other activity is occurring on the LAN? (Even though you think it is idle, is it really? There might be contention for available bandwidth. The contention might be a program you are using, or one that is using your LAN without your knowledge. The contention might be multiple machines in the same IP address.) As you said, the LAN speed is more than enough to support the playing of your media. The investigation continues. Joe L. Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.